ebook sql server dba training plan
TRANSCRIPT
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 138
copyBrent Ozar Unlimitedreg 2015 Page 1 htpBrentOzarcomneeds
How to Develop Your DBA CareerSQL Server DBA
Training Plan3rd Edition Spring 2015 - BrentOzarcomneeds
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 238
copyBrent Ozar Unlimitedreg 2015 Page 2 htpBrentOzarcomneeds
Yoursquore a developer or a sysadmin who wants to become a DBAWe know Wersquove been there too and wersquore here to share our lessons learned
so that your journey is easier than ours Wersquore Brent Ozar Unlimitedreg aconsulting company that makes SQL Server faster and more reliable
I learned to be a DBA thehard way The hard crappyway Our SQL Server was introuble and I was the kindof person who would roll up
my sleeves and 1047297gure outwhatever was broken Nextthing you know I was theone responsible for it
And boy did that suck
I didnt know about any free videos or blogsor e-books I didnt have thebudget to go to a class andeven if I did I didnt know whereto 1047297nd a good one DBA wasnt
in my job title and it wouldntbe for years
We want to make your learningexperience much beter thanmine was
This book - part of our free 6-Month DBA Training Plan -covers the basics of the topicplus link to our favorite free
training material on that topic
Your journey will start withOzarrsquos Hierarchy of DatabaseNeeds the pyramid shown atright In the next six months
well take you from the botomof the pyramid up to the top
You may not be able to 1047297x everything in yourenvironment during those six months but atleast youll understand the work involvedand how to con1047297dently get started
Weve collected thousands of resourcesover the years -training videos scripts blogposts about everythingfrom indexing to careers - and were excitedto share them with you Its all about makingyour journey to Professional Database
Administrator easier than ours was
Welcome to Your Training Plan
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 338
copyBrent Ozar Unlimitedreg 2015 Page 3 htpBrentOzarcomneeds
If You Have Questions
If you have questions about what yourereading start by Googling your questions It
sounds obvious but youd be amazed at howmuch good stuff there is out there to help(Im not being sarcastic This is exactly how
we get started whenever we have our ownquestions)
If you have a really short question and youexpect a really short answer and yoursquore onTwiter tweet the question with SQLhelp inyour tweet Lots of community membersmonitor this hash tag so even if they donrsquotfollow you personally theyrsquoll still see the
tweet and respond You can read more aboutSQLhelp too
If youd like to post a question tryDBAStackExchangecom or
SQLServerCentrals forums Yes both ofthese require registration but theyre totallyworth it On both of these sites thereshundreds - sometimes thousands - of peoplewho are itching to help answer your
questions They react fast too - make sureto go back and revisit your question every10-15 minutes for the 1047297rst few hours to seewhats happening Answer their clari1047297cation
questions and include as much detail as youcan For more instructions read Geting Help
with a Slow QueryIf you still cant get the answers you needemail us at HelpBrentOzarcom This is a
real email address manned by the realpeople at Brent Ozar Unlimited This isntone of those emails where it says Dont hitrespond because nobody cares Seriouslywe care and thats why we put these emailstogether Just please dont use that as your
FIRST resort - were real people with real jobs and real families and theres only somany hours per week that we can spendanswering questions By using the abovemethods 1047297rst youll be able to leverage thewhole communitys expertise instead of just
a few of us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438
copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds
Letrsquos start by making an Excel spreadsheet of the servers wersquove got the
number of databases on each one and over the coming weeks wersquoll 1047297x them
Build a Server Inventory
by Brent Ozar
At your company walk intothe VP of Saless officeand ask them how manysalespeople they have
NO I mean dont actuallyDO that because hes
going to ask you why the sales app is so
slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is
My point is that when youre in charge youneed to know exactly what youre in chargeof
Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for
bull SQL Server Version (2012 2008 2005)
bull Edition (Standard Enterprise Developer)
bull Environment (Production QAdevelopment disaster recovery)
bull Department (sales HR accounting ITmixed use)
bull Business Users Aff ected (list of people toemail when the server dies)
bull Application Names (internal or externalproduct names)
bull Plan B
That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our
Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about
Plan B
Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where
wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)
If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover
licensing as part of our training plan
What Well Do With This Spreadsheet
Right now you probably sleep well at night
thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add
more and more columns to this spreadsheetas we learn more about our environment
uncover problems and learn how to solvethem
For bonus points add a column for WhatScares Me Write a quick note about the one
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538
copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds
thing that scares youmost about this serverMaybe its blockingproblems maybe its the
failing jobs maybe itscode you dont
understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed
How to Survey Your Network for Servers
Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free
tools to do it
Microsof Assessment and Planning Toolkit -
its actually designed for licensingcompliance but it works great for building
server inventories It scans your networklooking for whatever programs you pick but
just con1047297ne it to SQL Servers only
Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your
network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin
privileges in the domain you might 1047297nd a lotmore servers than you expected
We dont get paid forplugging these productsand were always on thelookout for similar
inventory-building toolsso if you know of a beter
one email it to us atHelpBrentOzarcom
PSST Ask About This Before Geting Hired
When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security
concerns - but I just want to know if that listexists
Most of the timeit wont
This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list
right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this
email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has
to be stable and that means making surechanges dont happen accidentally
I created that sample support matrix when I
worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your
own Hope that helps
Most DBAs donrsquot
actually have a list of
their servers
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638
copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds
by Brent Ozar
The only reason we dobackups is so we can dorestores
When I 1047297rst started outas a SQL Server DBA Ithought things were
going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent
every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be
In theory you test our backup strategyahead of time with Kendras 5 Simple
Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers
In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its
just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a
database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier
Where to Do Restores
When youre restoring code (storedprocedures views triggers etc) or
individual tables dont restore onto the
production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a
diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments
Afer weve safely restored the right dataonto another server its easy to copy that
data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked
server tables
However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the
data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase
This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that
Bootcamps and
certi 1047297 cation
programs alone wonrsquot get
you that 1047297 rst jobmdash not when
your competition has
experience
(And yes this means wersquore assuming yoursquore backing them up)
The Reason We Back Up
All Those Databases
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738
copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds
particular server didnt have any otheractivity happening due to the outage Were
not always so lucky but it helps to think outof the box like that
A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that
youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase
Doing the Restore
Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to
apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore
When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or
transaction logbackups The goal is to
1047297nish the restorequickly
Next if diff erential
backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one
Next restore all of the transaction log
backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -
again using WITH NORECOVERYDoing all of this with the GUI sucks The
more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and
restores them for you automatically in orderWell talk about automating restores in the
next training module
To learn more about backups and restoresour favorite geting-started articles are
bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA
bull Brents DBA Nightmare SQL Down NoPlans
bull Jess 3 Things You Need to Start Doing toYour Database Server
When I Did My First Restore
I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and
early to print out the list of labels for the highschool graduates weddings babies and so
on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld
One morning as part of my
data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the
WHERE clause so Ideleted all of the photoshoots ever
Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the
restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it
Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested
Either the backups are
tested or yoursquore about
to be tested
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 238
copyBrent Ozar Unlimitedreg 2015 Page 2 htpBrentOzarcomneeds
Yoursquore a developer or a sysadmin who wants to become a DBAWe know Wersquove been there too and wersquore here to share our lessons learned
so that your journey is easier than ours Wersquore Brent Ozar Unlimitedreg aconsulting company that makes SQL Server faster and more reliable
I learned to be a DBA thehard way The hard crappyway Our SQL Server was introuble and I was the kindof person who would roll up
my sleeves and 1047297gure outwhatever was broken Nextthing you know I was theone responsible for it
And boy did that suck
I didnt know about any free videos or blogsor e-books I didnt have thebudget to go to a class andeven if I did I didnt know whereto 1047297nd a good one DBA wasnt
in my job title and it wouldntbe for years
We want to make your learningexperience much beter thanmine was
This book - part of our free 6-Month DBA Training Plan -covers the basics of the topicplus link to our favorite free
training material on that topic
Your journey will start withOzarrsquos Hierarchy of DatabaseNeeds the pyramid shown atright In the next six months
well take you from the botomof the pyramid up to the top
You may not be able to 1047297x everything in yourenvironment during those six months but atleast youll understand the work involvedand how to con1047297dently get started
Weve collected thousands of resourcesover the years -training videos scripts blogposts about everythingfrom indexing to careers - and were excitedto share them with you Its all about makingyour journey to Professional Database
Administrator easier than ours was
Welcome to Your Training Plan
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 338
copyBrent Ozar Unlimitedreg 2015 Page 3 htpBrentOzarcomneeds
If You Have Questions
If you have questions about what yourereading start by Googling your questions It
sounds obvious but youd be amazed at howmuch good stuff there is out there to help(Im not being sarcastic This is exactly how
we get started whenever we have our ownquestions)
If you have a really short question and youexpect a really short answer and yoursquore onTwiter tweet the question with SQLhelp inyour tweet Lots of community membersmonitor this hash tag so even if they donrsquotfollow you personally theyrsquoll still see the
tweet and respond You can read more aboutSQLhelp too
If youd like to post a question tryDBAStackExchangecom or
SQLServerCentrals forums Yes both ofthese require registration but theyre totallyworth it On both of these sites thereshundreds - sometimes thousands - of peoplewho are itching to help answer your
questions They react fast too - make sureto go back and revisit your question every10-15 minutes for the 1047297rst few hours to seewhats happening Answer their clari1047297cation
questions and include as much detail as youcan For more instructions read Geting Help
with a Slow QueryIf you still cant get the answers you needemail us at HelpBrentOzarcom This is a
real email address manned by the realpeople at Brent Ozar Unlimited This isntone of those emails where it says Dont hitrespond because nobody cares Seriouslywe care and thats why we put these emailstogether Just please dont use that as your
FIRST resort - were real people with real jobs and real families and theres only somany hours per week that we can spendanswering questions By using the abovemethods 1047297rst youll be able to leverage thewhole communitys expertise instead of just
a few of us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438
copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds
Letrsquos start by making an Excel spreadsheet of the servers wersquove got the
number of databases on each one and over the coming weeks wersquoll 1047297x them
Build a Server Inventory
by Brent Ozar
At your company walk intothe VP of Saless officeand ask them how manysalespeople they have
NO I mean dont actuallyDO that because hes
going to ask you why the sales app is so
slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is
My point is that when youre in charge youneed to know exactly what youre in chargeof
Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for
bull SQL Server Version (2012 2008 2005)
bull Edition (Standard Enterprise Developer)
bull Environment (Production QAdevelopment disaster recovery)
bull Department (sales HR accounting ITmixed use)
bull Business Users Aff ected (list of people toemail when the server dies)
bull Application Names (internal or externalproduct names)
bull Plan B
That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our
Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about
Plan B
Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where
wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)
If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover
licensing as part of our training plan
What Well Do With This Spreadsheet
Right now you probably sleep well at night
thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add
more and more columns to this spreadsheetas we learn more about our environment
uncover problems and learn how to solvethem
For bonus points add a column for WhatScares Me Write a quick note about the one
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538
copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds
thing that scares youmost about this serverMaybe its blockingproblems maybe its the
failing jobs maybe itscode you dont
understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed
How to Survey Your Network for Servers
Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free
tools to do it
Microsof Assessment and Planning Toolkit -
its actually designed for licensingcompliance but it works great for building
server inventories It scans your networklooking for whatever programs you pick but
just con1047297ne it to SQL Servers only
Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your
network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin
privileges in the domain you might 1047297nd a lotmore servers than you expected
We dont get paid forplugging these productsand were always on thelookout for similar
inventory-building toolsso if you know of a beter
one email it to us atHelpBrentOzarcom
PSST Ask About This Before Geting Hired
When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security
concerns - but I just want to know if that listexists
Most of the timeit wont
This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list
right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this
email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has
to be stable and that means making surechanges dont happen accidentally
I created that sample support matrix when I
worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your
own Hope that helps
Most DBAs donrsquot
actually have a list of
their servers
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638
copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds
by Brent Ozar
The only reason we dobackups is so we can dorestores
When I 1047297rst started outas a SQL Server DBA Ithought things were
going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent
every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be
In theory you test our backup strategyahead of time with Kendras 5 Simple
Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers
In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its
just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a
database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier
Where to Do Restores
When youre restoring code (storedprocedures views triggers etc) or
individual tables dont restore onto the
production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a
diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments
Afer weve safely restored the right dataonto another server its easy to copy that
data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked
server tables
However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the
data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase
This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that
Bootcamps and
certi 1047297 cation
programs alone wonrsquot get
you that 1047297 rst jobmdash not when
your competition has
experience
(And yes this means wersquore assuming yoursquore backing them up)
The Reason We Back Up
All Those Databases
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738
copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds
particular server didnt have any otheractivity happening due to the outage Were
not always so lucky but it helps to think outof the box like that
A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that
youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase
Doing the Restore
Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to
apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore
When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or
transaction logbackups The goal is to
1047297nish the restorequickly
Next if diff erential
backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one
Next restore all of the transaction log
backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -
again using WITH NORECOVERYDoing all of this with the GUI sucks The
more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and
restores them for you automatically in orderWell talk about automating restores in the
next training module
To learn more about backups and restoresour favorite geting-started articles are
bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA
bull Brents DBA Nightmare SQL Down NoPlans
bull Jess 3 Things You Need to Start Doing toYour Database Server
When I Did My First Restore
I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and
early to print out the list of labels for the highschool graduates weddings babies and so
on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld
One morning as part of my
data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the
WHERE clause so Ideleted all of the photoshoots ever
Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the
restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it
Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested
Either the backups are
tested or yoursquore about
to be tested
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 338
copyBrent Ozar Unlimitedreg 2015 Page 3 htpBrentOzarcomneeds
If You Have Questions
If you have questions about what yourereading start by Googling your questions It
sounds obvious but youd be amazed at howmuch good stuff there is out there to help(Im not being sarcastic This is exactly how
we get started whenever we have our ownquestions)
If you have a really short question and youexpect a really short answer and yoursquore onTwiter tweet the question with SQLhelp inyour tweet Lots of community membersmonitor this hash tag so even if they donrsquotfollow you personally theyrsquoll still see the
tweet and respond You can read more aboutSQLhelp too
If youd like to post a question tryDBAStackExchangecom or
SQLServerCentrals forums Yes both ofthese require registration but theyre totallyworth it On both of these sites thereshundreds - sometimes thousands - of peoplewho are itching to help answer your
questions They react fast too - make sureto go back and revisit your question every10-15 minutes for the 1047297rst few hours to seewhats happening Answer their clari1047297cation
questions and include as much detail as youcan For more instructions read Geting Help
with a Slow QueryIf you still cant get the answers you needemail us at HelpBrentOzarcom This is a
real email address manned by the realpeople at Brent Ozar Unlimited This isntone of those emails where it says Dont hitrespond because nobody cares Seriouslywe care and thats why we put these emailstogether Just please dont use that as your
FIRST resort - were real people with real jobs and real families and theres only somany hours per week that we can spendanswering questions By using the abovemethods 1047297rst youll be able to leverage thewhole communitys expertise instead of just
a few of us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438
copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds
Letrsquos start by making an Excel spreadsheet of the servers wersquove got the
number of databases on each one and over the coming weeks wersquoll 1047297x them
Build a Server Inventory
by Brent Ozar
At your company walk intothe VP of Saless officeand ask them how manysalespeople they have
NO I mean dont actuallyDO that because hes
going to ask you why the sales app is so
slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is
My point is that when youre in charge youneed to know exactly what youre in chargeof
Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for
bull SQL Server Version (2012 2008 2005)
bull Edition (Standard Enterprise Developer)
bull Environment (Production QAdevelopment disaster recovery)
bull Department (sales HR accounting ITmixed use)
bull Business Users Aff ected (list of people toemail when the server dies)
bull Application Names (internal or externalproduct names)
bull Plan B
That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our
Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about
Plan B
Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where
wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)
If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover
licensing as part of our training plan
What Well Do With This Spreadsheet
Right now you probably sleep well at night
thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add
more and more columns to this spreadsheetas we learn more about our environment
uncover problems and learn how to solvethem
For bonus points add a column for WhatScares Me Write a quick note about the one
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538
copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds
thing that scares youmost about this serverMaybe its blockingproblems maybe its the
failing jobs maybe itscode you dont
understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed
How to Survey Your Network for Servers
Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free
tools to do it
Microsof Assessment and Planning Toolkit -
its actually designed for licensingcompliance but it works great for building
server inventories It scans your networklooking for whatever programs you pick but
just con1047297ne it to SQL Servers only
Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your
network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin
privileges in the domain you might 1047297nd a lotmore servers than you expected
We dont get paid forplugging these productsand were always on thelookout for similar
inventory-building toolsso if you know of a beter
one email it to us atHelpBrentOzarcom
PSST Ask About This Before Geting Hired
When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security
concerns - but I just want to know if that listexists
Most of the timeit wont
This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list
right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this
email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has
to be stable and that means making surechanges dont happen accidentally
I created that sample support matrix when I
worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your
own Hope that helps
Most DBAs donrsquot
actually have a list of
their servers
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638
copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds
by Brent Ozar
The only reason we dobackups is so we can dorestores
When I 1047297rst started outas a SQL Server DBA Ithought things were
going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent
every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be
In theory you test our backup strategyahead of time with Kendras 5 Simple
Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers
In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its
just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a
database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier
Where to Do Restores
When youre restoring code (storedprocedures views triggers etc) or
individual tables dont restore onto the
production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a
diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments
Afer weve safely restored the right dataonto another server its easy to copy that
data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked
server tables
However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the
data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase
This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that
Bootcamps and
certi 1047297 cation
programs alone wonrsquot get
you that 1047297 rst jobmdash not when
your competition has
experience
(And yes this means wersquore assuming yoursquore backing them up)
The Reason We Back Up
All Those Databases
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738
copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds
particular server didnt have any otheractivity happening due to the outage Were
not always so lucky but it helps to think outof the box like that
A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that
youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase
Doing the Restore
Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to
apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore
When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or
transaction logbackups The goal is to
1047297nish the restorequickly
Next if diff erential
backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one
Next restore all of the transaction log
backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -
again using WITH NORECOVERYDoing all of this with the GUI sucks The
more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and
restores them for you automatically in orderWell talk about automating restores in the
next training module
To learn more about backups and restoresour favorite geting-started articles are
bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA
bull Brents DBA Nightmare SQL Down NoPlans
bull Jess 3 Things You Need to Start Doing toYour Database Server
When I Did My First Restore
I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and
early to print out the list of labels for the highschool graduates weddings babies and so
on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld
One morning as part of my
data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the
WHERE clause so Ideleted all of the photoshoots ever
Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the
restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it
Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested
Either the backups are
tested or yoursquore about
to be tested
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 438
copyBrent Ozar Unlimitedreg 2015 Page 4 htpBrentOzarcomneeds
Letrsquos start by making an Excel spreadsheet of the servers wersquove got the
number of databases on each one and over the coming weeks wersquoll 1047297x them
Build a Server Inventory
by Brent Ozar
At your company walk intothe VP of Saless officeand ask them how manysalespeople they have
NO I mean dont actuallyDO that because hes
going to ask you why the sales app is so
slow But I mean imagine if you COULD walkinto his office and ask him that I bet hewould have an instant answer He wouldntwait for a single moment Or walk into theCEOs office and ask how many employeeshe has Or ask the CFO how much the annualbudget is
My point is that when youre in charge youneed to know exactly what youre in chargeof
Make a Spreadsheet InventoryLets start by making a spreadsheet Acrossthe top make columns for
bull SQL Server Version (2012 2008 2005)
bull Edition (Standard Enterprise Developer)
bull Environment (Production QAdevelopment disaster recovery)
bull Department (sales HR accounting ITmixed use)
bull Business Users Aff ected (list of people toemail when the server dies)
bull Application Names (internal or externalproduct names)
bull Plan B
That last column gets a litle tricky - itmeans if this server dies in a 1047297re whats our
Plan B Are we going to restore thedatabases from another server Will we failover to a log shipped copy Or will weupdate our resume and head out for an earlylunch As we go farther into the trainingwere going to get much more speci1047297c about
Plan B
Theres no wrong answers here - week 1 isabout understanding where were at todaynot where wed like to be Were never where
wed like to be (Me personally Id like to beat a poolside bar right now but noooo Im ina hotel room waiting for my girlfriend to1047297nish blow drying her hair If youve everwondered why I write so much you can thankher full head of hair)
If youd like to get ambitious add additionalcolumns for Core Count CPU Count andMemory The core and CPU counts will getyou a head start on licensing although Ihave to confess that were not going to cover
licensing as part of our training plan
What Well Do With This Spreadsheet
Right now you probably sleep well at night
thinking you know everything thatshappening in these servers Hoooweee haveI got bad news for you Over the next sixmonths were going to progressively add
more and more columns to this spreadsheetas we learn more about our environment
uncover problems and learn how to solvethem
For bonus points add a column for WhatScares Me Write a quick note about the one
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538
copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds
thing that scares youmost about this serverMaybe its blockingproblems maybe its the
failing jobs maybe itscode you dont
understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed
How to Survey Your Network for Servers
Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free
tools to do it
Microsof Assessment and Planning Toolkit -
its actually designed for licensingcompliance but it works great for building
server inventories It scans your networklooking for whatever programs you pick but
just con1047297ne it to SQL Servers only
Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your
network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin
privileges in the domain you might 1047297nd a lotmore servers than you expected
We dont get paid forplugging these productsand were always on thelookout for similar
inventory-building toolsso if you know of a beter
one email it to us atHelpBrentOzarcom
PSST Ask About This Before Geting Hired
When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security
concerns - but I just want to know if that listexists
Most of the timeit wont
This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list
right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this
email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has
to be stable and that means making surechanges dont happen accidentally
I created that sample support matrix when I
worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your
own Hope that helps
Most DBAs donrsquot
actually have a list of
their servers
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638
copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds
by Brent Ozar
The only reason we dobackups is so we can dorestores
When I 1047297rst started outas a SQL Server DBA Ithought things were
going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent
every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be
In theory you test our backup strategyahead of time with Kendras 5 Simple
Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers
In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its
just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a
database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier
Where to Do Restores
When youre restoring code (storedprocedures views triggers etc) or
individual tables dont restore onto the
production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a
diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments
Afer weve safely restored the right dataonto another server its easy to copy that
data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked
server tables
However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the
data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase
This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that
Bootcamps and
certi 1047297 cation
programs alone wonrsquot get
you that 1047297 rst jobmdash not when
your competition has
experience
(And yes this means wersquore assuming yoursquore backing them up)
The Reason We Back Up
All Those Databases
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738
copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds
particular server didnt have any otheractivity happening due to the outage Were
not always so lucky but it helps to think outof the box like that
A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that
youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase
Doing the Restore
Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to
apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore
When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or
transaction logbackups The goal is to
1047297nish the restorequickly
Next if diff erential
backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one
Next restore all of the transaction log
backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -
again using WITH NORECOVERYDoing all of this with the GUI sucks The
more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and
restores them for you automatically in orderWell talk about automating restores in the
next training module
To learn more about backups and restoresour favorite geting-started articles are
bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA
bull Brents DBA Nightmare SQL Down NoPlans
bull Jess 3 Things You Need to Start Doing toYour Database Server
When I Did My First Restore
I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and
early to print out the list of labels for the highschool graduates weddings babies and so
on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld
One morning as part of my
data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the
WHERE clause so Ideleted all of the photoshoots ever
Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the
restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it
Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested
Either the backups are
tested or yoursquore about
to be tested
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 538
copyBrent Ozar Unlimitedreg 2015 Page 5 htpBrentOzarcomneeds
thing that scares youmost about this serverMaybe its blockingproblems maybe its the
failing jobs maybe itscode you dont
understand Six monthsfrom now I bet youll be proud of how thiscolumn has changed
How to Survey Your Network for Servers
Put a row in the spreadsheet for every serveryou have - whether youre in charge of it ornot We want to start with a good inventoryof what we have and theres two good free
tools to do it
Microsof Assessment and Planning Toolkit -
its actually designed for licensingcompliance but it works great for building
server inventories It scans your networklooking for whatever programs you pick but
just con1047297ne it to SQL Servers only
Quest Discovery Wizard for SQL Server - itsa GUI tool that pings all the servers in your
network and tries to 1047297gure out if theyve gotSQL Server installed If youre in a smallshop where your account has admin
privileges in the domain you might 1047297nd a lotmore servers than you expected
We dont get paid forplugging these productsand were always on thelookout for similar
inventory-building toolsso if you know of a beter
one email it to us atHelpBrentOzarcom
PSST Ask About This Before Geting Hired
When you take a new job as a DBA the very1047297rst question you should ask the company isDo you have a list handy of all the SQLServers Ill be managing I dont have to seethe list - I understand if you have security
concerns - but I just want to know if that listexists
Most of the timeit wont
This question serves two purposes it tellsYOU if the company has their act togetherwhen it comes to documentation and it tellsTHEM that youre the right person tomanage their database servers If they donthave the list theyre going to want that list
right away Nows your chance to explainhow you would go about gathering thatinformation (armed with the info in this
email)Bonus points create a SQL Server SupportMatrix a document that explains for yourdevelopers and end users whats allowed inproduction DR QA and development Thishelps set expectations going forward - if aservers going to be production then it has
to be stable and that means making surechanges dont happen accidentally
I created that sample support matrix when I
worked as a DBA and Ive shared it so youcan do a save-as and get a fast start on your
own Hope that helps
Most DBAs donrsquot
actually have a list of
their servers
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638
copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds
by Brent Ozar
The only reason we dobackups is so we can dorestores
When I 1047297rst started outas a SQL Server DBA Ithought things were
going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent
every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be
In theory you test our backup strategyahead of time with Kendras 5 Simple
Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers
In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its
just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a
database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier
Where to Do Restores
When youre restoring code (storedprocedures views triggers etc) or
individual tables dont restore onto the
production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a
diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments
Afer weve safely restored the right dataonto another server its easy to copy that
data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked
server tables
However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the
data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase
This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that
Bootcamps and
certi 1047297 cation
programs alone wonrsquot get
you that 1047297 rst jobmdash not when
your competition has
experience
(And yes this means wersquore assuming yoursquore backing them up)
The Reason We Back Up
All Those Databases
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738
copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds
particular server didnt have any otheractivity happening due to the outage Were
not always so lucky but it helps to think outof the box like that
A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that
youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase
Doing the Restore
Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to
apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore
When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or
transaction logbackups The goal is to
1047297nish the restorequickly
Next if diff erential
backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one
Next restore all of the transaction log
backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -
again using WITH NORECOVERYDoing all of this with the GUI sucks The
more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and
restores them for you automatically in orderWell talk about automating restores in the
next training module
To learn more about backups and restoresour favorite geting-started articles are
bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA
bull Brents DBA Nightmare SQL Down NoPlans
bull Jess 3 Things You Need to Start Doing toYour Database Server
When I Did My First Restore
I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and
early to print out the list of labels for the highschool graduates weddings babies and so
on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld
One morning as part of my
data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the
WHERE clause so Ideleted all of the photoshoots ever
Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the
restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it
Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested
Either the backups are
tested or yoursquore about
to be tested
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 638
copyBrent Ozar Unlimitedreg 2015 Page 6 htpBrentOzarcomneeds
by Brent Ozar
The only reason we dobackups is so we can dorestores
When I 1047297rst started outas a SQL Server DBA Ithought things were
going well as long as the backup jobs ransuccessfully Id go into SQL Server Agent
every now and then make sure they werestill running and that was the end of it I1047297gured if disaster ever struck Id just do arestore How hard could it be
In theory you test our backup strategyahead of time with Kendras 5 Simple
Questions About Your Backups and yoursquovememorized the 9 Leters that Get DBAsFired along with your companyrsquos answers
In practice small disasters strike all thetime The most common reasons to dorestores arent to revive an entire server - its
just to get back a few small tables or anindividual database Somebody ran thewrong DELETE statement or dropped a
database in production instead ofdevelopment and next thing you know wereall scrambling Lets think through a fewthings ahead of time to make the crisiseasier
Where to Do Restores
When youre restoring code (storedprocedures views triggers etc) or
individual tables dont restore onto the
production server I dont like touchingproduction servers more than I have to andlets face it - youre already having a badenough day as it is Thats why youre doing arestore remember So lets do our work on a
diff erent server (like dev or QA) and leaveproduction as it is Ive also writen aboutrestores in my ideal dev test and productionenvironments
Afer weve safely restored the right dataonto another server its easy to copy that
data across to other servers For simplicityand security you can set up a linked serveron the production box with read-only accessover to the restore server Then fromproduction you can run INSERT statementsusing a SELECT sourced from the linked
server tables
However if youre restoring tables over10GB youll probably want to do the restoresdirectly on the production server to make the
data copies faster Just make sure youreextremely careful with the scripting and thedatabase names - we dont want to restoreover the top of your working productiondatabase
This may require adding extra space to theproduction server In one emergency I freedup the necessary space by shrinking all ofTempDBs data and log 1047297les down to just1MB TempDB was on fast drives perfect fora one-time emergency restore and that
Bootcamps and
certi 1047297 cation
programs alone wonrsquot get
you that 1047297 rst jobmdash not when
your competition has
experience
(And yes this means wersquore assuming yoursquore backing them up)
The Reason We Back Up
All Those Databases
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738
copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds
particular server didnt have any otheractivity happening due to the outage Were
not always so lucky but it helps to think outof the box like that
A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that
youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase
Doing the Restore
Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to
apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore
When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or
transaction logbackups The goal is to
1047297nish the restorequickly
Next if diff erential
backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one
Next restore all of the transaction log
backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -
again using WITH NORECOVERYDoing all of this with the GUI sucks The
more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and
restores them for you automatically in orderWell talk about automating restores in the
next training module
To learn more about backups and restoresour favorite geting-started articles are
bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA
bull Brents DBA Nightmare SQL Down NoPlans
bull Jess 3 Things You Need to Start Doing toYour Database Server
When I Did My First Restore
I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and
early to print out the list of labels for the highschool graduates weddings babies and so
on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld
One morning as part of my
data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the
WHERE clause so Ideleted all of the photoshoots ever
Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the
restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it
Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested
Either the backups are
tested or yoursquore about
to be tested
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 738
copyBrent Ozar Unlimitedreg 2015 Page 7 htpBrentOzarcomneeds
particular server didnt have any otheractivity happening due to the outage Were
not always so lucky but it helps to think outof the box like that
A word of warning if referential integrity isinvolved like if youre trying to restore tablesthat have relationships to other tables that
youre NOT restoring then you can be in fora world of hurt here Were not going to coverthat scenario - it really is diff erent in eachcase
Doing the Restore
Restore the most recent full backup usingthe WITH NORECOVERY option - this isreally important This leaves the database ina restoring state so that you can continue to
apply additional backups to it If you forgetthose two key words your restore has tostart over again from scratch so please forthe love of all thats holy double-check thatoption before you start the restore
When Im restoring code or con1047297g tables thathavent changed since the last full backups Idont bother restoringany subsequentdiff erential backups or
transaction logbackups The goal is to
1047297nish the restorequickly
Next if diff erential
backups are involved restore the mostrecent diff erential WITH NORECOVERYDiff erential backups are cumulative - youonly have to restore the most recent one
Next restore all of the transaction log
backups afer the diff erential (or if you donthave diff s all of them afer the full backup) -
again using WITH NORECOVERYDoing all of this with the GUI sucks The
more backups you have the longer thistakes and the more likely you are to run intoerrors Instead what you need is a scriptthat looks at all of the backups in a folderplucks out the most recent relevant1047297les and
restores them for you automatically in orderWell talk about automating restores in the
next training module
To learn more about backups and restoresour favorite geting-started articles are
bull Grant Fritcheys SQL Server Backup andRestore for the Accidental DBA
bull Brents DBA Nightmare SQL Down NoPlans
bull Jess 3 Things You Need to Start Doing toYour Database Server
When I Did My First Restore
I did my 1047297rst emergency restore when I wasworking for a photo studio Id dropped out ofcollege and I took a job running theirdatabases Every morning I got in bright and
early to print out the list of labels for the highschool graduates weddings babies and so
on that were going to be photographed thatday The photographers would pick up theirstacks of labels for their 1047297lm (FILMremember that) and head out into the 1047297eld
One morning as part of my
data cleanup process I ranthe DELETE statementOne minor problem - Idforgoten to put in the
WHERE clause so Ideleted all of the photoshoots ever
Thankfully I was also the guy in charge ofbackups so afer I freaked out I put lastnights tape backups in and started the
restores When the boss came in to pick uphis stack of labels I was able to calmlyexplain what had happened and what I wasdoing to 1047297x it
Orvilles temper was legendary but I dodgeda bullet and kept my job Since thenwhenever I handle a database the very 1047297rstquestion I ask is Where are the backupsand when was the last time they weretested Either the backups are tested oryoure about to be tested
Either the backups are
tested or yoursquore about
to be tested
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 838
copyBrent Ozar Unlimitedreg 2015 Page 8 htpBrentOzarcomneeds
by Brent Ozar
If youre doing transactionlog backups forget using theGUI Even if you only haveone transaction log backupper hour itll take you way
too long to click through all the 1047297lesThink about what your backup folder mightlook like if we named our backup 1047297les bydatabase date time and a diff erentextension per type of backup (BAK for fullsDIF for diff erentials and TRN for transaction
logs)
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0100trn
bull MyDatabase_20130718_0200trnbull MyDatabase_20130718_0300trn
bull MyDatabase_20130718_0400trn
bull MyDatabase_20130718_0500trn
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
In that scenario I took my full backup at
midnight then hourlytransaction logswith diff erentialbackups every 6hours (This is nevera scenario Id use in
the real world but
its the simplest way to get the point acrossin a tiny email Hey you try teaching toughconcepts in a page or two buddy)
If disaster strikes at 815AM and I lose theserver I need to restore the full the mostrecent diff erential and the transaction logsthat follow the diff erential like this
bull MyDatabase_20130718_0000bak
bull MyDatabase_20130718_0600dif
bull MyDatabase_20130718_0700trn
bull MyDatabase_20130718_0800trn
Thats a really easy script to write - andthankfully MSSQLTips has already done itJust change the variables for your databasename and the path where your restore1047297les
live and presto the script will restore all ofthe 1047297les for you automatically
This Script Has Implications For You
You need to back up your databasesintelligentlyYou want to put each databasein its own folder and you want the 1047297le namesto have the database name and the time inthem Im a big fan of that anyway - it makeslife easier when I want to quickly scan and
see what backups have been done
It assumes the same databasepath when you restore If yourethe kind of DBA who likes to
change drive leters and foldersall the time or youve got 1047297lesscatered all over the place andyour production and
The GUI isnrsquot going to cut it Wersquore going to have to roll up our sleeves
As we script out a restore wersquoll learn how we need to design backups
Doing Faster Database Restores
Knowing how yoursquoll
restore aff ects how
you back up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 938
copyBrent Ozar Unlimitedreg 2015 Page 9 htpBrentOzarcomneeds
development database servers havediff erent drive leters then life is going to beharder for you There are other restorescripts that can adjust data and log 1047297le
names at restore time but theyre not quiteas elegant when it comes to restoring fulls
diff s and t-logsYou can do transaction log backups as ofenas you want If the restore process is fully
automated why not do transaction logbackups every minute It doesnt cost youany extra Rather than incurring extraoverhead it can actually incur LESSoverhead If youre only backing up logs oncean hour your users probably feel that hourly
load big time when it kicks off If you dosmaller backups more frequently theyll beless likely to notice the impact
You can automate your own 1047297re drill testing
Since this is done with plain old T-SQL youcan set up a SQL Agent job to restore lastnights backups onto your staging server Ilike seting up a DBA utility server chock fullof really cheap (but really large) SATAdrives like 2-4TB drives in a RAID 5 I have a
series of Agent jobs to restore a diff erentservers backups every day and then run
DBCC jobs on them This way I know Imgeting really good backups
Your Homework for This Chapter
You dont have to go build this wholeinfrastructure out - but start laying thegroundwork by making sure your backup1047297les are named in a restore-friendly way
Then try out the MSSQLTips restorescript to make sure you can quickly restore adatabase from scratch into a development or
staging environment If you get errors leavecomments on that blog post describing the
error and ideally contribute your scriptimprovements back in Now when disasterstrikes you wont be clicking around blindly
in a GUI - youll just open a proven script andhit Execute Bam
Geting Even Fancier
You can use this technique to build smallercopies of your production databases For
example at StackOver1047298ow I built an ultra-compressed backup with even more tricksOn my nightly restore server afer restoring
the production backups I dropped the non-clustered indexes rebuilt all objects with100 1047297ll factor and shrank the data and log1047297les down
The end result was a database that was over50 smaller
I then backed it up with compression and
lef
those backups in a folder that thedevelopers could access That made it easier
for developers to quickly grab a copy ofproduction data as of last night copy it totheir workstation as quickly as possible andrestore it faster with less drive spacerequired
Free 30-Minute Videos to Learn More
Log Shipping Part 1 Jes explains how logshipping is really just doing your restores
ahead of timeLog Shipping Part 2 when disaster strikeswhat yoursquoll need to do to fail over to thesecondary server and manage your jobs
Backup Planning Video Jeremiah explains 3
company scenarios and asks you to pick theright backup strategy for each
Our Backup Video Archive no itrsquos not abackup of our videos - itrsquos our videos about
backups We talk database mirroring
clustering AlwaysOn Availability Groupsand more
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1038
copyBrent Ozar Unlimitedreg 2015 Page 10 htpBrentOzarcomneeds
by Brent Ozar
Run our free sp_Blitztradestored procedure on one of
your production servers andpay particular atention tothe Security section of theresults It lists the logins
who have been granted thesysadmin or security admin roles
Dont think of them as logins
Think of them as people who can get you1047297red
These people can drop databases droptables change stored procedures edit dataor even change SQL Server con1047297gurationsetings like max server memory or maxdop
Youre probably not get
ing alerted when anyof these things change - we just cant aff ordto monitor every single thing in SQL Serverand send alerts on it At some point we haveto be able to trust certain logins and thatswhere the sysadmin and security admin rolescome in
Except when we 1047297rst get started learningdatabase administration its usuallybecause were the only DBA in the shop andthe server is a mess The front door is
unlocked the key is under the 1047298oor mat andeverybody knows weve got a big screen TVin the living room
How to Get Started Locking Things Down
Before you start removing peoples SArights be aware that there can be political
backlash In one shop the head developers
SA rights were removed and he stormedinto the DBAs office the next morningscreaming Turns out one of his appsautomatically created a processingdatabase every night did a bunch of work in
it and then dropped the database Nobody
knew because it was only around for 30-45minutes The problem could have been
avoided by communicating the securitychanges ahead of time and thats where we
need to startTake the sp_Blitztrade output to your manager -
just the security portions - and say
something like this
Heres the list of people who can doanything on the production server - deletedata drop databases or change performance setings If they do heres thelist of applications that will be aff ectedincluding accounting and payroll I dont
want to take away all of their permissions - I just want to start by giving them full permissions over their database but not inany other databases and not at the serverlevel Can I talk to them about doing that
Before we try to lock things down 1047297nd out who has copies of keys
Security Knowing Who Has
Access to the Servers amp Data
Think of sysadmin
logins as people
who can get you 1047297 red
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1138
copyBrent Ozar Unlimitedreg 2015 Page 11 htpBrentOzarcomneeds
Note that were only going to TALK to themnot actually do it because we need tocommunicate with them 1047297rst Then go to theend users or developers involved and say
Were starting to lock down the productionserver but I want to make sure you have all
the permissions you need Im going to makeyou a complete database owner inside yourdatabase so you can do anything you want inthere but Im going to take away your rightsto the other databases (like accounting and payroll) and Im going to remove your permissions to change server-level setingslike how much memory the server can useIm planning on doing it next weekend and Iwant you to have my email and phonenumber so that if anything breaks on thatdate you can call me and I can audit whatshappening on the server to see if its relatedto the permissions change
When You Get Resistance
When - not if - you get pushback fromdevelopers or users go to the projectmanagers or business people who have a
large stake in the database For example ifthe accounting database is on the server goto the CFO and say
Heres the list of people who can take downthe accounting system They have the permissions to drop the database at anygiven time and theres nothing I can do tostop it Id like to get that changed - can Ischedule a short meeting with you and thedevelopment manager to get everyone onthe same page
You want to turn it into a business problemnot a technology problem and the CFO will
very much be on your side She cant aff ordto have her entire department go down just
because some developer didnt put aWHERE clause on a T-SQL statement
I Know This Chapter Isnt Fun
Database administration isnt all bacon androses Sometimes its boring politics andpaperwork and this is one of those weeks
In the very 1047297rst chapter we built aspreadsheet inventory of our servers andnow its time to 1047297ll in a litle more details
Since were analyzing security we need toknow which applications live on each serverand whos in charge of each of those
applications You dont have to 1047297ll in thespeci1047297cs of who has read or writepermissions in each database but we want
Manage lots of
servers Learn
How to Be a
Senior DBA
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1238
copyBrent Ozar Unlimitedreg 2015 Page 12 htpBrentOzarcomneeds
to at least know the business purpose andthe business contact
The business contact is the one who really
helps us get the database locked downbecause their job is on the line if this data islost or unavailable (Come to think of it
youre someone who can get THEM 1047297red) Inthe coming weeks youll be working morewith them on reliability and performance
too so now is a good time to start 1047298eshingout that part of the spreadsheet
Database Administration is Politics
Sure we like to think were the police here toprotect and to serve but most of what we doinvolves siting in meetings convincingpeople to do what we want how we want
Its made more challenging because we ofendont have any real authority Sometimes theDBAs report to completely diff erentmanagers than the developers - and
sometimes its even diff erent companiesWe might be administering a database
server that houses a third-party applicationand the vendors support team demands tohave SA access
Consulting Lines
Afer consulting for a while I wrote a seriesof posts with lines you can use as a DBA
Sounds like youve got it all under control
What happens if that doesnt work
Would you mind driving
SQL Server needs a dog
High de1047297nition
video training on
your desktop
laptop or even
your iPad
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1338
copyBrent Ozar Unlimitedreg 2015 Page 13 htpBrentOzarcomneeds
by Brent Ozar
Download the SQL ServerCompliance Guide Its an
old 92-page whitepapercirca 2008 but it might justbe the best (and most
timeless) technical document that Microsof
has ever produced JC Cannon and DennyLee deserve a big thank-you for a great job
I only need you to read pages 7-13 In thosesix pages youll understand the diff erencesbetween risk management governance andcompliance Risk management meansknowing what risks the company is taking
with the data governance means the actionstaken to address the risks and compliancemeans someone is double-checking that
were actually doing the governance stuff wesay were doing
Your job as a DBA involves all three butwhen youre just geting started withcompliance focus on risk management Weneed to get a quick idea of the diff erent wayswe could lose data or that supposedly
secure data could get into the wrong hands(Yes your developers are probably thewrong hands but thats a diff erent story)
Homework Look for Risky BusinessOdds are nobody in the company has aninventory of the data were storing inridiculously unsafe ways Heres a quick wayto check your own database looking f ordangerous1047297elds
$ ()+-()+-$0+1$2+-
30) $2+--+ 4 567899lt=65
) $2+--+ 4 569gt865
) $2+--+ 4 56gtltA=B65
Feel free to change out those keywords for
other terms that are relevant to yourbusiness - 1047297elds that are sensitive and thatwould hurt if they got out into the wild Thenlook at the contents of those tables - is thedata being stored unencrypted Who hasaccess to it
If were storing unencrypted passwords inthe database for example then everydatabase backup weve ever done isdangerous If anybody gets access to anybackup 1047297le like our off site tape backupsthen we could be on the front page of the
news tomorrow
When you 1047297nd it email the developers andthe business contact for that databaseExplain that the data is not encrypted and
use examples from the SQL ServerCompliance Guide to show how this is a riskfor the business
Compliance Knowing Who Accesses What
Its unusual for me to assign homework 1047297rstin the email and then go on to talk aboutother things but I want you to be aware ofother things that companies usually doaround their secure data
Top tip when you play Risk go for South America Itrsquos only 4 countries
with 2 access points so itrsquos easy to defend See security maters
Risk isnrsquot just a board game
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1438
copyBrent Ozar Unlimitedreg 2015 Page 14 htpBrentOzarcomneeds
Ofen management will say We need toaudit everyone who changes or accessessecure data Technically SQL Server hasfeatures that can accomplish this goal -
things like SQL Server Auditing In practicethough this is a problem for hard-core
security teamsbecause the verysame DBA whomanages the
servers also haspermissions tochange the SQLServer audits Agreedy DBA couldeasily disable
auditing get the necessary data and then
enable it again
With seriously ambitious work you can lockauditing down so that cant happen but its really really hard
As a result most seriously securecompanies end up with a completelydiff erent auditing solution that lives outsideof the DBAs realm The security team buys
third party hardware appliances like IBMGuardium or Imperva that act like a network
1047297rewall between everyone and the SQLServer The appliance logs everything thathappens in the SQL Server and then thesecurity team can review those logs without
you (the DBA) knowing about it
These solutions are typically six 1047297gures andup Theyre expensive because they have tobe absolutely bulletproof - if they fail youcant have the access to the database
suddenly stop Plus youll need them inplace for not just your productionenvironment but also your disaster recoveryenvironment
If you just want to check a box and make theauditors think youre secure thats easy and
cheap but seriously good security isseriously expensive
Big Companies Love Security
The bigger the company the more interestedthey get in security
In small shopswithout a dedicatedsecuritydepartmenteverybody is ofen adomainadministrator and
everybody knowswhere all the
passwords are stored
In big shops with a dedicated security teamthe security team usually makes requests to
disable the DBAs sysadmin access Theywant the DBA to only have enoughpermissions to do their job but not enoughpermissions to see the database contents(Afer all the DBA shouldnt see credit carddata health data etc)
This isnt a realistic request in SQL Server2012 and prior but SQL Server 2014 bringsnew server roles that will make this request
easier For more info check out the securityenhancements section of Whats New in2014
Learning More About Security
Check out Denny Cherrys book Securing
SQL Server (paperback - Kindle) Dennymanages to bring a very dry topic to life
Every shop with PCI HIPAA SOX or othercompliance needs should de1047297nitely own a
copy of this book Its a tougher sell for smallshops with only one full time DBA thoughbecause a lot of the security featuresdiscussed take time to digest andimplement
Know whether your
company just wants
to check a box on an auditorrsquos
form or really be secure
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1538
copyBrent Ozar Unlimitedreg 2015 Page 15 htpBrentOzarcomneeds
by Brent Ozar
Wersquove coveredbackups and securityexcept
Well we havenrsquot
We could go on forhundreds of pages about those twolevels alone We could 1047297ll entire ebooksperhaps even libraries with coolconcepts like how the diff erentialbitmap works how to secure stored
procedures with certi1047297cates how tocon1047297gure SQL Server when usingVMware-based backups for storagereplication how to recover from acorrupted nonclustered index yadda
yadda yaddaOur point of this ebook is a fast start
Your journey in SQL Server databaseadministration is never really done Yoursquore
going to be learning for the rest of your lifeand even when you think yoursquore done yoursquoreonly mistaken Heck Irsquom a Microsof Certi1047297ed Master and I still learn things inalmost every presentation I atend (ExceptJeremiahrsquos because Irsquom not smart enough
to digest that stuff )So wersquore going to move up a level tocapacity but please donrsquot email us with criesof complaints about how you wish we would
have gone deeper into clustering setupsreplication internals or whatever other
Trivial Pursuit category yoursquore hot on thisweek We just canrsquot cover it all in one ebook
When you 1047297nd an area that you want to digdeeper into check out
Our video training courses - from our $29
90-minute courses all the way up to our 5-6hour monsters at $299 we can go muchdeeper Some even include quizzes
Our in-person classes - we have 2-3 daycourses for developers and DBAs who need
to make SQL Server faster and morereliable
Our SQL Critical Caretrade - we work togetherwith you over WebEx or in person todiagnose your SQL Serverrsquos pain points and
train you on the 1047297xes
In Ozarrsquos Hierarchy of Database Needs wersquove touched on the botom two
layers backups and security Before we move up a level letrsquos pause
Itrsquos Time to Level Up
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1638
copyBrent Ozar Unlimitedreg 2015 Page 16 htpBrentOzarcomneeds
by Brent Ozar
Donrsquot
Itrsquos not your job
No seriously hang on asecond Im not saying that
you should ever say things like Its not my job but I do want you to understand whenyou should avoid reinventing the wheel YourWindows administrators should bemonitoring every Windows server you havemaking sure it doesnt run out of drive space
This isnt some wild unique need that onlySQL Server has - its everywhere in your datacenter
(You might actually even BE a Windows
admin just here because you need to learn
about working with SQL Server It mightactually BE your job to monitor this stuff Thats cool - thats how I got started too)
In our journey from the base of our Hierarchy
of Database Needs to the top we do indeedneed to talk about capacity - but I dont wantto monitor drive space from inside SQLServer and I dont want to buy SQL-speci1047297ctools in order to pull it off Yes you cantechnically use commands like
xp_ 1047297xeddrives to list all of the SQL Serversdrive leters and how much free space theyhave but that doesnt work everywhere Forexample you might have mount points ordatabases on UNC paths neither of whichwill show up in xp_ 1047297xeddrives So leave the
drive space monitoring to the pros
Why You Shouldnt Build a Monitoring Tool
If you want to be a professional developeryou should build things You should learn
what exceptions could happen how to trapthem and how to fail elegantly Its hardwork but if you get good at it - really good -you can build amazing things and make a
killer living for yourself
But if you want to be a professional DBAyou should leave the developing to thedevelopers
Im not saying you shouldnt learn thedynamic management views (DMVs) how to
dig through them to 1047297nd the evidence youneed and how to turn that data into actionsHopefully Im going to teach you how to do alot of that over the course of the next six
months Take another look at the Hierarchyof Database Needs again and think for asecond about all the things were going to belearning over the next six months Just in thelast 1047297ve weeks Ive had you build aninventory of your servers automate yourdatabase restores start locking down
security and enumerate your database risksThe next few months are chock full of thingsthat will help you become a hero in the eyesof your users and your developers
Building a crappy monitoring tool in your
spare time will not give you that same levelof respect (And yes if youve only got yourspare time at work your monitoring tool isgoing to be crappy If youve got so muchtime that you can build a great monitoring
Just when you thought it was safe to let your guard down
we throw a trick chapter at you
How to Monitor Drive Space
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1738
copyBrent Ozar Unlimitedreg 2015 Page 17 htpBrentOzarcomneeds
tool you need to focus on providing resultsto the business fast because youre about tobe laid off as pure overhead)
How to Pick a Monitoring Tool
Theres basically three kinds of monitoring
tools out there for SQL Server
Hardware Alerting - when you buy a serverfrom big companies like Dell HP or IBMthey come with built-in management toolsThese tools are surprisingly powerful - forexample check out our post on How to Use
HP System Management Homepage
Updown Alerting - these tools make surethe SQL Server service is running and that ithas all the basic needs taken care of If the
server runs out of drive space if the servicestops if the event log starts 1047297lling up witherrors or zombies atack these tools will letyou know The most popular sofware in thiscategory is Quest Spotlight Idera SQLDiagnostic Manager and Red Gate SQLMonitor
Performance Diagnostics - these tools try toderive insight from SQL Servers dynamicmanagement views (DMVs) to help
performance tuners wring the mostthroughput out of their hardware The mostpopularsofware hereis SQL SentryPerformanceAdvisor
QuestFoglightPerformanceAnalysis and
to someextent the
tools in theabovecategory too
If I was you Id start by asking the Windowsteam if theyve got any sofware that handles
the updown alerting for services drivecapacity monitoring etc If so get them tostart monitoring your servers Im being
sel1047297sh here - my real reason is that I want tosave my monitoring budget for tools in thePerformance Diagnostics category
Surprisingly all of these tools are around thesame price - around $1000-$2000 USDper monitored instance
Next itrsquos time to evaluate them Donrsquot justinstall a bunch of random tools in productionand see what they 1047297nd Instead no materwhich category of tool youre buying make alist of the last 4-5 things that have caused
your phone to ring afer hours Heres someof my personal favorites
bull Deadlocks
bull Out-of-control query running for hours
bull Long-running job like slow backup
bull Queries that desperately need an index
bull SQL Server not accepting connections
Figure out how to recreate those samesymptoms in your developmentenvironment and then get a free trial of a
couple of the tools I mentioned (All of themprovide free 10-14 day trials) Reproduce the
problem and watch howthe monitoring sofwarereacts Does it lead youto the root cause
quickly or does it justhave a bunch of 1047298ashingred lights on theconsole The best oneswill save you time by
get
ing you right to thesolution
Afer youve done yourevaluation and picked a
favorite get a quote from them - and get a
quote from the other vendors as well Sharethe quotes with the competing salespeople
If yoursquore building a
monitoring tool in your
spare time at work itrsquos either
going to be crappy or yoursquore
about to be laid off because you
have too much spare time
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1838
copyBrent Ozar Unlimitedreg 2015 Page 18 htpBrentOzarcomneeds
Youd be surprised how willing they are tonegotiate and you can use the cheapestquote to get the tool you really want
Your Homework
I dont expect you to magically get budget
approval for a tool this week but I want youto start making a list of day to day problemsas they strike If you waste four hours
troubleshooting a deadlock problem make anote of the date the time required and a fewwords about the root cause In a month ortwo youll be able to present this list to yourboss as proof of why the right monitoringtool could save you money Down the road
Ill also link you to a video showing you howto get the budget approval for that tool
Im not selling you vendor tools by the way Iofen link to vendor tools but I dont receive
any compensation Im just a huge believerin using the right tool to look like a hero fast- instead of building your own hammer everytime you change jobs
How This Philosophy Cost Me a Job
I actually failed a DBA job interview once forgiving the same answer in my email here Thelead DBA asked me If you were going to
monitor your servers how would you do it Isaid Id buy a tool off the shelf and he wasfurious He wanted me to use an open sourcemonitoring framework
Today several years down the road now thatIm a Microsof Certi1047297ed Master my answer
is still the same I dont reinvent the wheeland neither should you Focus on things thatprovide the business tremendous value andyour career will take care of itself
If your business would1047297nd tremendousvalue in a killer monitoring system thenbecome a developer and give it to them orconsider contributing to the huge number ofopen source monitoring products Dontbuild a new one from scratch and de1047297nitelyknow which metrics to monitor
Bobcats per 100 Orders andOther Spurious Metrics
by Jeremiah Peschka
Did you know that you canship a bobcat 130th of the
time and still maintain 97positive feedback on ebay
What other statistical lies are lurking outthere for you to 1047297nd in Perfmon Cache HitRatio Disk Queue Length Page LifeExpectancy Page Splits and UserConnections can be bogus Learn why
Monitoring SSDPerformance
by Jeremiah Peschka
What if we could watch SSDwear in real time Manyvendors off er SMART status
codes to return detailedinformation about the status of the driveRotational drives can tell you how hot thedrive is provide bad sector counts and ahost of other information about drive healthRead more in this post
Before you build
or buy check out
the best free
SQL Server
downloads
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 1938
copyBrent Ozar Unlimitedreg 2015 Page 19 htpBrentOzarcomneeds
While I donrsquot want you reinventing the ldquoIs the C Drive Fullrdquo wheel
I very much want you to know whatrsquos going on inside each database
Manage Space Inside Databases
by Brent Ozar
Inside each of yourdatabase data 1047297les(MDFs) SQL Server storesyour data in 8KB pages
Thats kilobytes - notmegabytes not gigabytes
but just mere kilobytes
Say that we create a table$) C =D1E7FAA9
GE7FAAH - H-IGJKJLK
E7FAA-8EA M)$0)GNOOLL
First off yes I understand I shouldnt doEmployeeName in one 1047297eld - I should haveFirstName LastName MiddleName SuffixPre1047297x yadda yadda yadda but Im trying to
keep this email short Now see what you did
Its long again Doggone it its hard teachingthis stuff in an email
Anyhoo in this table each record takes upjust a litle bit of space EmployeeID is an
INTeger which takes 4 bytes Its the same 4bytes whether that number is 1 or1000000 EmployeeName is aVARCHAR(200) which means we can storeup to 200 characters in here and each
character takes a byte If we insert BRENTOZAR thats 10 characters (and boy am I a
character) so we need 10 bytes to store it
If all of our employees average about 10characters in their name that means we
could 1047297t about 500-600 records per 8KBdatabase page (In reality theres someoverhead because SQL Server also needs to
use some parts of the page to storemetadata and well talk about that later in
the training)
Brent Ozar Unlimited is a small company sowe can keep all of our employees on a single8KB page As we insert update and deleteemployees SQL Server fetches that 8KB
page off disk brings it into memory makesthe necessary changes and then writes that
data page back to disk The 8KB page itselfis the smallest unit that SQL Server willcache - it doesnt cache individual rows
records - and each page belongs exclusivelyto just one object
A Word About Objects
Youll notice that I avoid using the word
table Tables are cool but as we start to diginto what SQL Servers doing under thehood we want to start thinking about these
three object typesHeap - a table with no clustered index In mydboEmployees table I didnt specify in whatorder SQL Server should store my data soits just going to slap the data down on my8KB page in any old order
Clustered Index - what we normally think ofas a table If Id have created my table likethis
$) C =D1E7FAA9
GE7FAAH - H-IGJKJL P)+)I4I $2)HK
E7FAA-8EA M)$0)GNOOLL
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2038
copyBrent Ozar Unlimitedreg 2015 Page 20 htpBrentOzarcomneeds
Then SQL Server would store my data inorder of EmployeeID That way when Isearch for EmployeeID 42 it can godirectly to that number without scanning
through all of my employees The clusteredindex is sorted in the order of the
EmployeeID1047297eld but its actually the fullcopy of our table including all of our 1047297elds -in this case just EmployeeName
Nonclustered index - If I tell SQL Server to
$) --$2)H -HQ Q-8EA
- =D1E7FAA9GE7FAA-8EAL
Then SQL Server will create a second copyof my table sorted by EmployeeName Thiscopy of my table will
only include the1047297elds speci1047297ed in my
index(EmployeeName)plus whatever 1047297eldsit needs to get backto the clustered index(in this case my
clustering keyEmployeeID)
All three of these objects - heaps clustered
indexes and nonclustered indexes - will bestored on separate sets of pages We wonthave the clustered index and nonclusteredindex for the same table on the same page -theyre split Thats why when were doingspace analysis we have to think in terms ofindexes not tables To learn more index
basics read Jes Borlands SQL Server IndexTerms
Pages amp Rows on Disk
The sysdm_db_index_physical stats Dynamic Management Function (DMF)returns the number of rows and pages storedin each database object It takes parametersfor database ID and object ID or you canpass in NULLs to get information across all
of your database objects Scroll down to theexamples link in that Books Online pageand youll get queries to examine a singletable - Id strongly recommend starting with
a small table because some of theparameters for this DMF will cause SQL
Server to actually look at every page in theobject That means if all of the pages for thatobject arent cached in memory SQL Serverwill go pull those pages off disk and that can
slow down your running SQL Server
This DMF also includes average record sizeand max record size This makes for funspelunking how big is each record reallyJust because we make everything a
VARCHAR(8000)doesnt mean wereactually storing 8000characters in each 1047297eldNow dont go changingyour database structure
just yet - you can easilybreak applications whendatatypes change Letsleave that for later
You can get similar metadata much faster byusing sp_spaceused but it doesnt get the
cool info about average record size and Iwanted to encourage you to go spelunkinghere
Learning More About Pages
In my 90-minute session How to Think Likethe Engine I explain pages indexes joinsSARGability and more I use real databasepages from the StackOver1047298owcom
database for demos and you actually getPDFs to print out and follow along as we go
Your Homework
Lets start thinking about what objects aretaking up space in our databases Check outour free sp_BlitzIndextrade stored procedurethat analyzes the indexes in your database
Armed with thisknowledge of
pages yoursquoll make bet er
data modeling decisions
too (MAX) ainrsquot free
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2138
copyBrent Ozar Unlimitedreg 2015 Page 21 htpBrentOzarcomneeds
from a psychologists point of view Is yourdatabase a hoarder clutching on to a bunchof nonclustered indexes that arent getingused and are just taking up space The
details columns in the results will show howbig each index is and whether its geting
used or notThis is important because the more indexesyou have
bull The longer your backups take
bull The longer index rebuilds take
bull The more memory you need to cacheeverything
And most importantly the slower yourinsertsupdatesdeletes go because SQL
Server has to maintain more copies of yourtable
As we start to move up the Hierarchy ofNeeds from capacity into performance youllstart to see how these foundational items
are so important
Want to See What a Page Looks Like
Check out the DBCC PAGE command Youpass in a database name 1047297le number and
page number and SQL Server will return thenearly-raw contents of that page along withdiagnostic information Its a fun way to get apeek under the hood
This topic is a good example of how knowingthe basics of database internals can come inhandy when you step back and think aboutdatabase server performance You donthave to use DBCC PAGE as part of your jobas a DBA but just knowing what an 8KB
page is helps you understand the output of
various Dynamic Management Views(DMVs) when they report back units inpages
Wersquove got tons of
blog posts and
videos about our
favorite indextips tricks and
free tools
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2238
copyBrent Ozar Unlimitedreg 2015 Page 22 htpBrentOzarcomneeds
If everythingrsquos stored on an 8KB page and each page has only one object on
it then we can 1047297gure out what objects (tablesindexes) are geting cached
What Pages are In Memory
by Brent Ozar
We like to think SQLServer is using all of ourmemory to cache data butthats just part of it SQL
Server uses memory forlots of things
bull Caching database objects
bull Sorting data for your query results
bull Caching execution plans
bull Performing system tasks
Ofen were surprised by how litle data isbeing cached for each database
Last section we looked at the 8KB pages inour database Those pages are the same
whether theyre on disk or in memory - theyinclude the database ID and the object ID so
if we looked at all of the pages in memory wecould 1047297gure out which tables are beingcached in memory right now The belowquery gives us the magic answers but be
aware that the more memory you have thelonger this will take It wont block otherusers but it could take a minute or two ifyouve got gt64GB memory several minutesif youve got a terabyte or more
$ $G$2-GL R S JONT1O -2+)$GJOK NLL $8gtUA=H8B8+C K
$ =8B8D89A= 30- VNWXW 0-
5)A9YltgtAHD5 HC-+G=8B8D89A=L-H H8B8D89A-8EA
()+ 9F91=E9DYZZAlt=A9gtlt7Blt9
[)2P CI HC-+G=8B8D89A=L K
=8B8D89A=
)H) CI J H$
Compare the size of each database versushow much is being cached Ofen in the 1047297eld
Ill see 100GB databases that just have8-12GB of data cached in memory Thatmight be completely okay - if you onlyregularly query just that amount of data - butwhat if we constantly need all 100GB andwere constantly pulling it from disk
This Leads to Cool Questions
This DMV query leads to so many coolperformance tuning questions I get so
excited by these concepts
How fast are the cached pages changing From the moment we read an 8KB page off disk how long does it stay in memory beforewe have to 1047298ush it out of the cache to make
room for something else were reading off disk This concept is Page Life Expectancya Perfmon counter that measures in secondshow long things stay in RAM The longer thebeter as I explain in my Perfmon tutorial
Do the results change based on time of day This is a one-time snapshot of whats inmemory at the moment but it can change in
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2338
copyBrent Ozar Unlimitedreg 2015 Page 23 htpBrentOzarcomneeds
a heartbeat If you have automatedprocesses that run a bunch of reports in asingle database at 2AM then the memorypicture will look completely diff erent then
Are we caching low-value data If you mixvendor apps and in-house-writen apps on
the server youll ofen 1047297nd that the worst-writen application will use the most memoryThing is that might not be the most
important application Unfortunately wedont have a way of capping how muchmemory gets used by each database This iswhy most shops prefer to run vendorapplications on separate virtual machines orservers - this way they dont hog all the
memory on a SQL Server that needs to serveother applications
Do we have enough memory If yourerunning SQL Server 2008R212 Standard
Edition youre limited to just 64GB ofphysical RAM If youre running SQL Serveron bare metal (not a VM) and youve got anyless than 64GB go buy enough to get to64GB Its the safest easiest performancetuning change you can make If youre in a
VM or running Enterprise Edition thememory question gets a lot tougher To learn
more read A Sysadmins Guide to SQLServer Memory
Are we using memory for anything otherthan SQL Server If weve got IntegrationServices Analysis Services ReportingServices or any other applications installedon our server these are robbing us ofprecious memory that we might need tocache data Dont remote desktop into your
SQL Server and run SSMS either - its amemory pig Put your management tools ona virtual machine in the data center andremote desktop into that instead
Can we reduce memory needs with indexesIf weve got a really wide table (lots of 1047297elds)or a really wide index and were not actively
querying most of those 1047297elds then werecaching a whole bunch of data we dontneed Remember SQL Server is caching atthe page level not at the 1047297eld level A
nonclustered index is a narrower copy of thetable with just the 1047297eldscolumns we want
The less 1047297elds the more data we can pack inper page The more we can pack in the moredata were caching and the less we need tohit disk
When I tune indexes on a server Ive neverseen before sysdm_os_buff er_descriptorsis one of the 1047297rst places I look The databasewith the most stuff cached here is likely to bethe one that needs the most index help
Itrsquos Probably Not a SAN Problem
When I was a junior DBA I focused a lot onthe storage I kept complaining to my SAN
administrators because my storage didntrespond fast enough - my drives were taking50ms 100ms or even 200ms in order todeliver data for my queries
The SAN admin kept saying Its okay The
SAN has a cache Thing is the size of theSANs cache is typically 32GB-128GB -which at 1047297rst sounds like a lot - but divide it
between all of the servers connected to theSAN Ofen we 1047297nd that an individual SQLServer might get only a couple of gigabytes
of SAN cache Thats way less than what theSQL Server has in memory What are theodds that when we need data for a query itsnot going to be in SQL Servers 64GB ofmemory but it IS going to be in the SANsmiserly 2GB of cache Not gonna happen
SAN caching is still great for writesespecially for the transaction log but dont
count on it helping for SELECT speeds
To learn more check out our training courseon Storage Virtualization and Hardware forDBAs
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2438
copyBrent Ozar Unlimitedreg 2015 Page 24 htpBrentOzarcomneeds
by Brent Ozar
For our evil trainingpurposes lets say we workfor the phone company andwe need a database tablewith phone numbers We
need to track
bull Phone number (required)
bull Billing contact last name (required)
bull Billing contact 1047297rst name (required)
bull Business name (optional)
bull Business category (restaurant doggroomer auto dealer etc)
bull Address 1
bull Address 2
bull City
bull State
bull Zip
bull Service start date
(Sometimes a person or a business will havemultiple phone numbers but for the sake of
this training lets keep it a simple 1047298at table)We will never have two records in here with
the same phone number We have to tell ourdatabase about that by making the phonenumber our primary key
When we make the phone number theprimary key were telling SQL Server thatthere can be no duplicate phone numbers
That means every time a record is inserted orupdated in this table SQL Server has tocheck to make sure nobody exists with thatsame phone number As of the year 2000there were about 360000 people in Miami
Throw in businesses and lets say our tablehas 500000 records in it
That means by default every time we insertone eensy litle record SQL Server has to
read half a million records just to make surenobody else has the same phone numberEvery 1 write = 500000 reads Well thatwont work will it So lets organize our tablein the order of phone number That waywhen SQL Server inserts or updates
records it can quickly jump to the exact areaof that phone number anddetermine whether or not theresany existing duplicates This iscalled seting up a primaryCLUSTERED key Its called
clustered because - well I have noidea why its called clustered but
the botom line is that if you could look at theactual hard drive the data was stored on itwould be stored in order of phone number
Now we have the table organized by phonenumber and if we want to 1047297nd people by
phone number itll be very fast While ourcomputer systems will usually need to grabpeoples data by phone number our
customers and end users ofen need to getnumbers by other ways Thats where non-clustered indexes come in
In which we pretend the phone company but instead of giving everybody
unlimited calling we just organize the data
Indexes What Goes First
No Brentrsquoshome number
is not in this chapter
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2538
copyBrent Ozar Unlimitedreg 2015 Page 25 htpBrentOzarcomneeds
The White Pages A Non-Clustered Index
Our customers constantly need to 1047297nd
peoples phone numbers by their name Theydont know the phone number but they knowthe last name and 1047297rst name We would
create an index called the White Pagesbull Billing contact last name
bull Billing contact 1047297rst name
bull Phone number
That index would save people a ton of timeThink about how you use the white pages
1 You scan through pages looking at justthe leters at the top until you get close
2 When you get close you open up the fullbook and jump to the right leters
3 You can quickly 1047297nd the right single one
record
Now think about how you would do it withoutthe White Pages Think if you only had abook with 500000 records in it organizedby phone number You would have to scan
through all 500000 records and check thelast name and 1047297rst name 1047297elds Thedatabase works the same way except itseven worse If a developer wrote a SQLquery looking for the phone number it wouldlook like this
$ PUA-YEDAlt ()+ HltAgtBltF 30)89B-8EA ] 5EBU5 -H (lt9B-8EA ]5^U5
That doesnt say select the top one - it says
select ALL of them When you as a humanbeing go through that list of 500000 phonenumbers you would stop when you thoughtyou found the right John Smith Thedatabase server cant do that - if it 1047297ndsJohn Smith at row 15 it doesnt materbecause there might be a few John Smiths
Whenever you do a table scan and you dontspecify how many records you need itabsolutely positively has to scan all
500000 records no mater what
If the database has an index by last name
and 1047297rst name though the database servercan quickly jump to Smith John and startreading The instant it hits Smith
Johnathan it knows it can stop becausetheres no more John Smiths
Covering FieldsHelping Indexes Help You
But thats not always enough Sometimes wehave more than one John Smith and thecustomer needs to know which John Smith
to call Af
er all if your name was JohnSmith and the phone book didnt include
your address youd get prety tired ofanswering the phone and saying No youwant the John Smith on Red Road Hes305-838-3333 So we would add theAddress 1 1047297eld in there too
bull Billing contact last name
bull Billing contact 1047297rst name
bull Address 1
bull Phone number
Do we absolutely need the address in ourindex for every query No but we include it
for convenience because when we DO needit we need it bad And if we DONT need it itdoesnt really hurt us much
This is called a covering index because itcovers other 1047297elds that are useful Adding
the address 1047297eld to our index does make itlarger A phone book without addresseswould be a litle thinner and we could packmore on a page We probably dont want toinclude the Address 2 1047297eld because theAddress 1 1047297eld is enough to get what we
need The database administrator has tomake judgement calls as to which 1047297elds touse on a covering index and which ones to
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2638
copyBrent Ozar Unlimitedreg 2015 Page 26 htpBrentOzarcomneeds
skip When building covering indexes thecovering 1047297elds go at the end of the index
Obviously this index would suck
bull Billing contact last name
bull Address 1
bull Billing contact 1047297rst name
bull Phone number
We dont want all of the Smiths ordered bytheir address and then a jumbled mess of
1047297rst names That wouldnt be as fast andeasy to use Thats why the covering1047297elds goat the end and the names go 1047297rst - becausewe use those
Selectivity Why the Last Name Goes First
If you wanted to search for Brent Ozar in thephone book you look in the Os for Ozar 1047297rstand then youll 1047297nd Ozar Brent This is moreefficient than organizing the phone book by
1047297rst name then last name because there aremore unique last names than 1047297rst namesThere are probably more Brents in Miamithan Ozars This is called selectivity The last
name 1047297eld is more selective than the 1047297rstname 1047297eld because it has more uniquevalues
For lookup tables - meaning when usersneed to look up a speci1047297c record - whenyouve narrowed down the list of 1047297elds that
youre going to use in an index generally youput the most selective 1047297eld 1047297rst
Indexes should almost never be set up with anon-selective 1047297eld 1047297rst like Gender Imaginea phone book organized by Gender LastName First Name it would only be usefulwhen you wanted a complete list of allwomen in Miami Not that thats a bad thing -
but no mater how much of a suave guy youthink you are you dont really need ALL of
the women in Miami This is why non-selective indexes arent all that useful onlookup tables
This rule is really important for lookuptables but what if you arent trying to look upa single speci1047297c record What if youreinterested in a range of records Well letslook at
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2738
copyBrent Ozar Unlimitedreg 2015 Page 27 htpBrentOzarcomneeds
The Yellow Pages Another Index
When we need to 1047297nd a dog groomer wedont want to go shuffling through the white
pages looking for anything that sounds like adog groomer We want a list of organized bybusiness category
bull Business Category
bull Business Name
bull Address 1
bull Phone Number
Then well look at the list of businesses seewhich name sounds the coolest and whichaddress is closest to ours and well call afew of them Well work with several of the
records Here were searching for a range ofrecords not just a single one
Notice that we didnt put the most selective1047297eld 1047297rst in the index The 1047297eld BusinessName is more selective than Business
Category But we put Business Category1047297rst because we need to work with a range ofrecords When youre building indexes younot only need to know what 1047297elds areimportant but you have to know how theuser is fetching records If they need several
records in a row next to each other then itmay be more helpful to arrange the recordslike that by carefully choosing the order ofthe 1047297elds in the index
Learning More About Indexes
Indexes are really important so well becovering these in more depth in the next twoemails In the meantime heres a few greatresources on geting started with indexes
Our index resources page - where weve gotposts and videos about heaps indexing fordeletes partitioning and more
Our Indexing videos - free 30-minute videoson indexing mistakes DBA Darwin Awardsand how to design smarter indexes with theDMVs
SQLServerCentrals Index Stairway - a 15-part series by David Durant that goes all theway to indexing internals
Expert Performance Indexing by JasonStrate and Ted Krueger (book) - covers howindexes work and how to pick the right one
Also available on Kindle
And 1047297nally if yoursquod like a video training
session with our very own Microsof Certi1047297edMaster Kendra Litle wersquove got a 6-hour setof videos complete with quizzes and demos
How to Tune
Indexes and
Speed Up SQL
6-Hour Training
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2838
copyBrent Ozar Unlimitedreg 2015 Page 28 htpBrentOzarcomneeds
by Brent Ozar
Last time we talked aboutthe two most common typesof indexes - clustered andnonclustered In this weeksepisode were going to
spend just a paragraph or two covering theother types of indexes starting withcovering indexes HA Get it Were coveringcovering indexes Oh I kill me
Covering Indexes
Covering indexes arent actually a diff erentkind of index - its a term that is used incombination with a query and an index If Ihave this query
$ (lt9B-8EAK 89B-8EAK PUA-YEDAlt()+ =D1PA7A 30) 89B-8EA ] 5_8lt5
And if I have this index
$) -HQ Q89B-8EAgtY=A9 -=D1PA7A G89B-8EAL -$2H
G(lt9B-8EAK PUA-YEDAltL
Then the index covers all of the 1047297elds I needto run this query SQL Server will start bylooking up all of the Ozars by last name andthen the index includes the FirstName andPhoneNumber1047297elds SQL Server doesnt
have to go back to the clustered index in
order to get the results I need This meansfaster queries plus less contention - itleaves the clustered index (and the othernonclustered indexes) free for other queriesto use
Covering indexes are most eff ective whenyou have very frequent queries that
constantly read data and theyre causingblocking problems or heavy IO
Filtered Indexes
Say were a big huge online store namedafer a river and we constantly add recordsto our dboOrders table as customers place
orders We need to query orders that haventbeen processed yet like this
$ lt=Alt-YEDAlt ()+ =D1lt=Alt930) lt=AltPltgtA99A= ] O
The vast majority of the Orders records willhave OrderProcessed = 1 because we keepall of our order history in this table If wecreate an index on the OrdersProcessed1047297eld its going to have a lot of data - butwere never going to run queries looking for
OrderProcessed = 1 Starting with SQL
Server 2008 we can create an index with aWHERE clause
$) -HQ Qlt=AltB8BY9 -=D1lt=Alt9 Glt=Alt-YEDAltL 30)
lt=AltPltgtA99A= ] O
Covering1047297ltered full text XML heaps 3x5 index cards you name it
More Kinds of Indexes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 2938
copyBrent Ozar Unlimitedreg 2015 Page 29 htpBrentOzarcomneeds
Now we have an index on just a few records -
a small subset of our table
Full Text Indexes
Lets say we have a table calleddboMoviePlots and it had a Description1047297eld where we put each movies plots Weknow we liked this one movie where a guywas afraid of snakes but we couldntremember the exact table We could write aquery that says
$ ()+ =D1+`APB9 30)HA9gtlt7B 4 5698aA965
But that wouldnt be veryefficient SQL Serverwould have to look at
every moviesdescription and scrollthrough all of the wordsone character at a time looking for snakesEven if we index the Description 1047297eld werestill going to have to scan every row
Full text indexes break up a text 1047297eld likeDescription into each word and then storesthe list of words in a separate index Theyreblazing fast if you need to look for speci1047297c
words - but only as long as you rewrite yourquery to use the full text search commandslike this
$ ()+ =D1+`APB9 30)$--GHA9gtlt7BK 598aA95L
You can even do fun stuff like look forsynonyms or variations on a word To learnmore about full text indexing check out
bull Books Online on Full Text Indexing -seriously stop laughing the manuals
really good This link is for SQL 20142012but keep in mind that there were changesfrom 2008 to 2012
bull Understanding Full Text Indexing by RobertSheldon - covers SQL Server 2008 plus
the diff erences between 2005 and 2008(which were huge)
XML Indexes
You can store XML data natively in SQLServer tables using XML 1047297elds SQL Serveris aware of the contents - in the sense that
SQL Server knows the content is valid XML -but its not necessarily smart aboutsearching the data
When you run XML queries SQL Server has
to roll up its sleeves and parse the XML dataEvery Single TimeThats CPU-intensiveand a recipe for slowperformance Insteadwe can create pre-
processed versions of
the XML so we can rapidly jump to speci1047297cnodes or values
bull Primary and Secondary XML Indexes -
Books Onlinebull Selective XML Indexes - instead of wasting
a ton of space indexing values we neveruse SQL Server 2012 can create theequivalent of 1047297ltered indexes on XML
Heaps
Heaps are tables with no clustered indexwhatsoever Theyre tables stored in randomorder data slapped in any old place that 1047297ts
When you want to query a heap SQL Server
scans the whole freakin thing Every SingleTime
Sounds bad right Most of the time it is -except for a couple of very niche uses If you
have a log-only table meaning theres insertsbut never any updates deletes or selectsthen a heap can be faster If you have a
Why did it have
to be snakes
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3038
copyBrent Ozar Unlimitedreg 2015 Page 30 htpBrentOzarcomneeds
staging table in a data warehouse where youshove a lot of data in quickly and then needto get it all out at once then delete all of it aheap can be faster Just make sure you test
it to make sure its actually faster under yourapplications needs
Picking the Right Indexes for Your Apps
SQL Servers Dynamic Management Views
(DMVs) surface a lot of usefulinstrumentation like which indexes aregeting used which ones arent geting usedand which ones SQL Server wishes it wouldhave had Unfortunately they dont give youa holistic overall picture - theyre just raw
data that has to be manually combined andinterpreted Well talk about that in coming
lessons but you need to have this groundknowledge of index options 1047297rst
Dude Who Stole My
Missing Index
Recommendation
by Kendra Litle
Recently Jes asked the team
an index tuning question ldquoIf a query has anindex hint in it will the optimizer eversuggest a missing index for that queryrdquo
I immediately loved the question because Irsquodnever really thought about it before Itypically think of index hints as being a veryrisky game and avoid them whenever I canndashafer all if someone drops the index yoursquovehinted any query hinting a non-existent
index will start to fail (Thatrsquos a really badday)
Even so some people love index hints
Read the full story wwwbrentozarcomarchive201307dude-who-stole-my-missing-index-recommendation
How to Master Index
Tuning in One Step
by Kendra Litle
Irsquom going to tell you a secret
Index tuning is complicated but itrsquossomething you can become great at You justneed to practice it regularly
Herersquos that one step stop thinking indextuning is a problem for Future You
Thatrsquos it Really If you read this headline anddidnrsquot skip the post your job probablyinvolves helping an application using SQLServer go faster If thatrsquos the case indextuning is a problem for Present You Itrsquos a
problem for you now itrsquos a problem for younext month and itrsquos still a problem the monthafer that
Index tuning isnrsquot something you do once a
year Itrsquos something that you need to doiterativelyndash that means every month Overtime data sizes change user activitychanges and the SQL Server optimizerchanges Each of these things mean thatindexes that are best for an application will
alsochange As you tune indexes your query
plans will change and yoursquore very likely tosee more opportunities to add drop andcombine indexes emerge Because of thisyou want to do a few changes every month
I hear a lot of reasons why people donrsquot tunetheir indexes
Read the full story wwwbrentozarcomarchive201308how-to-master-sql-server
index-tuning
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3138
copyBrent Ozar Unlimitedreg 2015 Page 31 htpBrentOzarcomneeds
by Kendra Litle
Once up on a time there wasa database server with500GB of data and a heavyread workload of dynamicqueries Data was updated
frequently throughout the day and indextuning was a serious challenge At the bestof times performance was dicey
Things went bad
Application performance plummeted Lots of
code changes had been released recentlydata was growing rapidly and the hardwarewasnt the absolute freshest There was nosingle smoking gun-- there were 20 smokingguns
A team was formed of developers and ITstaff to tackle the performance issue Earlyin the process they reviewed maintenance onthe database server Someone asked aboutindex fragmentation The DBA Manager
said Of course were handlingfragmentation But a few queries were runand some large seriously fragmentedindexes were discovered in production
The DBA explained that fragmentation
wasnt the problem He didnt haveautomated index maintenance set up buthe periodically manually defragmentedindexes that were more than 75fragmented
Bad meet ugly
The whole performance team 1047298ipped out
Trust disappeared Managers squirmedMore managers were called in
The DBA tried to change the subject but it
was just too late More than a week waswasted over Fragmentation-Gate It was ahuge embarrassing distraction and itsolved nothing
Heres the deal-- the DBA was actually right
Fragmentation wasnt the root cause of theperformance problem But he made amiscalculation he should have set upoccasional automated index maintenance toalign with his teams normal practices andstandards
Why you need automated indexmaintenance
When performance gets bad one of the very1047297rst things people look at is whethersystems involved are con1047297gured accordingto best practices If youre not following abest practice you need a good reason why
Regular index maintenance still has a lot of
merit even in Shangri-La where your dataall 1047297ts into memory and your storage systemis a rockstar with random IO indexmaintenance can help make sure that youdont have a lot of empty space wasting
loads of memory
Its still a good idea to automate indexmaintenance Dont go too crazy with it--monitor the runtime and IO use and run itonly at low volume times to make sure it
helps more than it hurts
Indexes are like cars You have to maintain them yoursquore probably not doing
it and if you take them to a mechanic yoursquoll probably get overcharged
The Parable of Index Maintenance
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3238
copyBrent Ozar Unlimitedreg 2015 Page 32 htpBrentOzarcomneeds
How much downtime canyou spare
Before you implement
index maintenance 1047297ndout how much time tablescan be offline in each of
your databases
If youve got SQL Server
Standard Edition indexrebuilds are alwaysoffline
Even with SQL Server
Enterprise Edition youcan specify an onlinerebuild unless the indexcontains large object types (This
restriction is relaxed somewhat in SQLServer 2012)
Partitioned tables are especially tricky Youcan rebuild an entire partitioned indexonline but partition level rebuilds are offlineuntil SQL Server 2014
Maintenance plans or custom scripts
You can go the easy way and use SQL ServerMaintenance Plans but unfortunately
theyre very simplistic you can only sayrebuild all the indexes or reorganize all theindexes You cannot say If the index is45 or more fragmented rebuild it--otherwise do nothing If you dont spendmuch time with SQL Server and youve gotdowntime available every weekend this can
be a decent option
If you need to minimize downtime customindex maintenance scripts are the way to goOur favorite Ola Hallengrens maintenance
scripts These are super 1047298exible welldocumented and hellip free The scripts have allsorts of cool options like time boxing andstatistics maintenance
Download and con1047297gure them on a test
instance 1047297rst Theres a lot of options on
parameters and youllneed to play with them
Get used to the cmdexec
job step types When youinstall the scripts youllsee that the SQL Server
Agent jobs run indexmaintenance using a callto sqlcmdexe in an
MSDOS style step Thatsby design
Use the examples on thewebsite If you scroll tothe botom of the index
maintenance page youll1047297nd all sorts of examples showing
how to get the procedure to do diff erentuseful things
Find out when maintenance fails
Dont forget to make sure that yourmaintenance jobs are successfully loggingtheir progress Set up Database Mail andoperators so jobs let you know if they fail
Tell your boss you did a good thing
Finally write up a quick summary of what you
did why you chose custom scripts ormaintenance plans and why Share it withyour manager and explain that youve set upautomated index maintenance as a proactivestep
Having your manager know youre taking the
time to follow best practices certainly wonthurt-- and one of these days it just mighthelp you out
Learning More About Fragmentation
5 Things About Fill Factor - Including whatits for what its NOT for and why youshouldnt play with that
Stop Worrying About Fragmentation -
defragging everything can cause moreproblems than it solves
Our Best Free
SQL Downloads
includes videotutorial on Olarsquos
script setup
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3338
copyBrent Ozar Unlimitedreg 2015 Page 33 htpBrentOzarcomneeds
Time to 1047297nd out if yoursquove been reading or just ldquoscanningrdquo
Backup amp Recovery Questions
1 How many production SQL Servers doyou have
2 Whats the RPO and RTO of your mostimportant production server
3 Did your backups take the normalamount of time last night
4 When was the last time DBCCsuccessfully 1047297nished in production
Security Questions
1 How many diff erent people aresysadmins in production
2 Do they each know that theyresysadmins and take care to avoidaccidents
3 How many of your databases hold
personally identi1047297able data like creditcard numbers social security numbersand passwords
4 If someone gets hold of one of those
database backups can your companysdata go public
5 Have you informed your managers ofthat risk or will they blame you
Monitoring Questions
1 When a database server runs out of drive
space who gets emailed
2 Do at least two diff erent people get theemail in case one is on vacation orunavailable
3 What actions will you take to1047297x thesituation
4 Are those actions documented so thateveryone who gets the email can take
action quickly
Index Questions
1 Does every table in production have aclustered index
2 For any exceptions (heaps) do you havea plan to 1047297x them
3 Which table in production has the most
indexes and why
4 Which frequently queried tables inproduction have the least indexes andwhy
5 How are you managing indexfragmentation
Pop Quiz
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3438
copyBrent Ozar Unlimitedreg 2015 Page 34 htpBrentOzarcomneeds
The Right Answers
Theres no one right answer forany of these questions but
some answers are more wrongthan others Database administration is a
journey and not everyone in the company is
going to appreciate the work youre putingin You can spend weeks or months trying toimprove your answers on these No mater
how big your company is and how manydatabase administrators you have youreprobably never going to be truly happy withyour answers here
Youre not aiming for perfect
Youre aiming for good enough that yourmanagers accept the base of your Database
Hierarchy of Needs pyramid and that youfeel con1047297dent in tackling the higher levels
like performance and future-proo1047297ng
Next weeks email is going to start digginginto performance and were going to ignorethe lower levels of the pyramid - but thatdoesnt mean that part of your journey is
over Print out this email cut out thequestion list and scribble in a few thoughts
Pin it up on your wall and a few months fromnow when youre feeling overcon1047297dent thatyour environment is awesome check that list
again Refresh your memory with the links onthe right side of this email
When an outsider comes in like a supportengineer or a consultant theyre going tostart at the base of your pyramid 1047297rst Before
they start to help they have to make sureyour data is backed up and checked for
corruption They cant go making changeswithout having a safety net
And you shouldnt either
Remember that as we start talking aboutchanging server and database setings next
Donrsquot make changes
without a tested safety net
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3538
copyBrent Ozar Unlimitedreg 2015 Page 35 htpBrentOzarcomneeds
As SQL Server runs queries it constantly tracks what it waits on
Ask SQL Server for these wait statistics and tuning is easy Easier anyway
What is SQL Server Waiting On
by Brent Ozar
You probably got intodatabase administrationby way of development orsystems administration
Youre used to monitoringstuff from the OUTSIDE
using things like Performance Monitorcounters
SQL Server has a way way way beter tool
When SQL Server starts running your queryyour query consumes CPU It sits on a CPUscheduler using as much CPU as it can allto its greedy self SQL Server doesnt carve
up a core and say you can run for 3 secondsuntil someone else gets to run - oh no Yourquery runs until its done burning CPU thewhole time
Until it has to wait for somethingThe instant your query has to wait - like ifSQL Server needs to read data from harddrives or wait for someone else to let go of alock - then your query steps off the CPU and
goes into a waiting queue SQL Servertracks how many milliseconds your queryspends waiting and what resource itswaiting on
The Crappy Way to Check Waits
Run this simple query
$ ()+ 9F91=E98B9B8B9
And youll get back a list of wait types plushow many milliseconds the server has spent
waiting on this wait type The time iscumulative measured over time since theSQL Server instance was started - or sincesomeone manually cleared the wait stats(Dont do that)
Theres a few problems here the wait list isreally cryptic its 1047297lled with irrelevant systemwait types and its measured over timeWhat you really want is a quick snapshot ofwaits over a 30-second period with thesystem wait types 1047297ltered out
The Beter Way to Check Waits
Hit BrentOzarcomgowaitsnow and get ourfree script This returns 3 result sets - the
1047297rst showing how long the server has beenup the second showing the waits since theywere last cleared and the third is a fun oneThe third shows a running sample of waitsover the last 30 seconds
If your server is busy youll see MORE than30 seconds of waits on the biggestbotlenecks Thats totally normal becauseyour SQL Server has multiple cores each ofwhich may have multiple queries lined upwaiting for something
If your server isnt busy your waits willprobably add up to 30 seconds - or much
You might be surprised at
how li t le memory is
used for caching data
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3638
copyBrent Ozar Unlimitedreg 2015 Page 36 htpBrentOzarcomneeds
less This just means the servers mostlysiting around idle Dont make bigperformance tuning decisions based onsmall samples like that - aim for sampling
when the servers really busy
To learn more about wait types and what
they mean check out our wait typesresources page
Once you get started with wait stats youllwant to capture this data all the time andtrend it Dont reinvent that wheel everymodern performance monitoring tool trackswait stats already
Correlating Waits Stats and Perfmon
Once we think weve got a botleneck we
need to double-check those numbers bygathering server-level metrics about that
particular botleneck
In our Performance Monitor tutorial weexplain how to set up Perfmon gather theright metrics and export them to aspreadsheet Depending on the wait stats
youre seeing as a botleneck heres thePerfmon counters to collect
To double-check CXPACKET and
SOS_SCHEDULER_YIELD waits collectSystem Processor Queue Length(for each individual core not the total)
This wait type indicates challenges withparallelism Parallelism isnt a bad thing - it
means SQL Server is breaking out your largequeries into multiple tasks and spreadingthat load across multiple processors Learnmore about CXPACKET waits
For PAGEIOLATCH and WRITELOG waits
Physical Disk Avg SecRead
Physical Disk Avg SecWrite
Physical Disk Avg ReadsSec
Physical Disk Avg WritesSec
The top two counters are about responsetime - how fast the storage is returningresults The botom two counters are abouthow much work were giving to the storage
Much like you the more work youre asked todo the slower you get The top two are the
SAN persons fault the botom two are yourfault You want to make sure the botom twonumbers trend down over time by doingbeter indexing
You need to 1047297gure
out how to help an
ailing SQL Server
Our SQL Critical
Caretrade helps
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3738
copyBrent Ozar Unlimitedreg 2015 Page 37 htpBrentOzarcomneeds
For LCK_ waits collect
SQLServer Locks - Lock Waitssec
SQL Server Locks - Avg Wait Time
SQL Server Access Methods - Table LockEscalationssec
SQL Server Transactions - Longest RunningTransaction Time
SQL Server Access Methods - Full Scanssec
These counters help you determine howmuch locking is going on and where thesource is For example when youre havinglocking problems and the Full Scanssecreports a high number maybe youve got a
lot of table scans going on and those aregrabbing locks across tables while theywork Or maybe Longest RunningTransaction Time is reporting a few minutes -meaning someone is running a really longtransaction and its starting to block lots of
other users
Got Other Waits
Wait types can be so cryptic - therersquos somany of them and theyrsquore ofen not
documented well To learn more about waittypes and what they mean check out ourwait types resources page
Learning More About Perfmon Counters
No mater what your biggest wait type is theidea here is to correlate that wait type withunderlying Perfmon counter measurementsto drill down deeper and 1047297nd the root causeof the problem To learn more heres our
favorite resources
Perfmon Counters of Interest Poster - fromQuest Sofware listing the best counters by
typeOur Perfmon tutorial - explaining how tocollect the data and analyze it
Jimmy Mays Perfmon Workbook - an Excelspreadsheet with Jimmys favorite counters
and thresholds
Watch Brent explain wait stats while hersquos
dressed up as Richard Simmons Click here
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us
8182019 eBook SQL Server DBA Training Plan
httpslidepdfcomreaderfullebook-sql-server-dba-training-plan 3838
by Brent Ozar Unlimitedreg
We love - no we LOVE - helpingpeople get relief for data pains
and wersquove got lots of options tohelp
FIRST AID TOTALLY FREE STUFF
We build cool troubleshooting tools and givethem away for free
bull sp_Blitzreg - fast SQL Server health check
bull sp_BlitzIndexreg - identi1047297es indexingmadness dragging down your SQL Server
bull Our blog - thousands of articles onperformance tuning availability andcareer development
bull Much more - like our posters YouTubevideos and weekly webcasts
IN-PERSON TRAINING CLASSESACROSS THE UNITED STATES
Our classes are taught by real experts withhands-on knowledge - speci1047297cally us Weshare the latest cuting-edge tips and tricksthat wersquove learned in real-life deployments
Wersquore available for questions and answers -itrsquos your chance to talk face-to-face and getpersonal advice on your tough challenges
Join us in-person at our classes
VIDEO COURSES$29-$299 FOR IN-DEPTH KNOW-HOW
bull How to Think Like the SQL Server Engine$29 - Kick start your performance tuningwith insight into SQL Serverrsquos brain
bull DBA Job Interview Question and AnswerKit $29 - Practice questions and more
bull Virtualization SANs and Hardware for
SQL Server $299 - 5 hours of subsystemsecrets
bull How to Tune Indexes and Speed Up SQLServer $299 65 hours of quizzesscripts and more
You can watch our high-de1047297nition trainingfrom your desktop laptop or even your iPad
for 18 months
Thatrsquos just a sample - check out the full list
SQL CRITICAL CAREreg
A FASTER SAFER SERVER IN 4 DAYS
Tired of struggling with a slow unreliableSQL Server
In just 4 days wersquoll work together with you toget to the root cause explain your optionsand give you a simple prioritized action planto make the pain stop
We donrsquot keep secrets you get to keep our
scripts and you watch us work Itrsquos like thebest conference training but in your ownenvironment
Schedule a free 30-minute consultation with
us and learn more about our SQL CriticalCarereg email HelpBrentOzarcom
Wersquove got more tricks than a pony
Get More Help from Us