oleg natalushko. drupal server anatomy. drupalcamp kyiv 2011

Drupal Server Anatomy

Presented by IT Patrol inc.

Gold Sponsor ofDrupalCamp Kyiv 2011

Silver Sponsors ofDrupalCamp Kyiv 2011

• Should I look for another hosting company or it is time for a dedicated server• How to choose hosting company for

dedicated server• How to choose dedicated server for your

needs• What next ?

Plan

Possible “Overload” Reasons

• Bots indexing your website• DDOS attack• Habr-effect• Drupal modules with known

performance issue• The problem from theme part of drupal• Your website actually grew up and need

more resources

Bots Access Log Example

*** [11/Apr/2011:03:39:09 +0400] ******* Googlebot/2.1

*** [11/Apr/2011:03:39:10 +0400] ******* bingbot/2.0

*** [11/Apr/2011:03:39:10 +0400] ******* Googlebot/2.1

*** [11/Apr/2011:03:39:10 +0400] ******* YandexBot/3.0

*** [11/Apr/2011:03:39:10 +0400] ******* msnbot/2.0b

*** [11/Apr/2011:03:39:11 +0400] ******* YandexBlogs/0.99

*** [11/Apr/2011:03:39:12 +0400] ******* Yahoo! Slurp

*** [11/Apr/2011:03:39:12 +0400] ******* YandexAntivirus/2.0

Casper Bot Search Alexibot Art-Online asterias BackDoorbot Black.Hole BlackWidow BlowFish botALot BuiltbotTough Bullseye BunnySlippers Cegbfeieh Cheesebot CherryPicker ChinaClaw CopyRightCheck cosmos Crescent Custo DISCo DittoSpyder DownloadsDemon eCatch EirGrabber EmailCollector EmailSiphon EmailWolf EroCrawler ExpresssWebPictures ExtractorPro EyeNetIE FlashGet Foobot FrontPage GetRight GetWeb! Go-Ahead-Got-It Go!Zilla GrabNet Grafula Harvest hloader HMView httplib HTTrack humanlinks ImagesStripper ImagesSucker IndysLibrary InfonaviRobot InterGET Internet Ninja Jennybot JetCar JOC Web Spider Kenjin.Spider Keyword.Density larbin LeechFTP Lexibot libWeb/clsHTTP LinkextractorPro LinkScan/8.1a.Unix LinkWalker lwp-trivial Mass Downloader Mata.Hari Microsoft.URL MIDown tool MIIxpc Mister.PiX Mister PiX moget Mozilla/3.Mozilla/2.01 Mozilla.*NEWT Navroad NearSite NetAnts NetMechanic NetSpider Net Vampire NetZIP NICErsPRO NPbot Octopus Offline.Explorer Offline Explorer Offline Navigator Openfind Pagerabber Papa Foto pavuk pcBrowser Program Shareware 1 ProPowerbot/2.14 ProWebWalker ProWebWalker psbot/0.1 QueryN.Metasearch ReGet RepoMonkey RMA SiteSnagger SlySearch SmartDownload Spankbot spanner Superbot SuperHTTP Surfbot suzuran Szukacz/1.4 tAkeOut Teleport Teleport Pro Telesoft The.Intraformant TheNomad TightTwatbot Titan toCrawl/UrlDispatcher toCrawl/UrlDispatcher True_Robot turingos Turnitinbot/1.5 URLy.Warning VCI VoidEYE WebAuto WebBandit WebCopier WebEMailExtrac.* WebEnhancer WebFetch WebGo IS Web.Image.Collector Web Image Collector WebLeacher WebmasterWorldForumbot WebReaper WebSauger Website eXtractor Website.Quester Website Quester Webster.Pro WebStripper Web Sucker WebWhacker WebZip Wget Widow [Ww]eb[Bb]andit WWW-Collector-E WWWOFFLE Xaldon WebSpider Xenu's Zeus

List of Bots that is NOT important

#block bots

RewriteCond %{HTTP_USER_AGENT} Aport [OR]

RewriteCond %{HTTP_USER_AGENT} Ezooms [OR]

RewriteCond %{HTTP_USER_AGENT} Yahoo [OR]

RewriteCond %{HTTP_USER_AGENT} TwitterFeed [OR]

RewriteCond %{HTTP_USER_AGENT} MJ12bot [OR]

RewriteCond %{HTTP_USER_AGENT} rss [OR]

RewriteCond %{HTTP_USER_AGENT} YandexBot [OR]

RewriteCond %{HTTP_USER_AGENT} bingbot

RewriteRule ^.* - [F,L]

Bots Blocking Rule in .htaccess

DDOS Attack

Habr-effect(Slashdot effect)

Bad Code Example

?

< 1000 visits/day < 5 req/min, < 1000 nodes < 30 req/min, < 1000 nodes, 90% bots

Should I look for other hosting provider?

?

Network infrastructure, speed guarantees, UPS, backup power, diesel generators, Guarantees replacement time, The qualification of the technical support.

Important Parameters of the Data Center

?

CPUMemoryHard driveBandwidth

Checklist for Choosing Didicated Server

SATA IIVelociraptorSASSCSISSD

HDD Types

RAID0RAID1RAID5RAID10

RAID Types

RAID 1 RAID 5

300 GB SAS

300 GB SAS

1 TBSATA II

1 TBSATA II

1 TBSATA II

1 TBSATA II

Architecture of Our HDD Subsystem

We are using

NGINX and Apache interrelation diagram

How “opcode cache” works

“OpCode cacher” performance diagram

High-performance MySQL server

Corporate website: http://it-patrol.ru

Drupal Hosting: http://dh.it-patrol.ru

Contact email: support@it-patrol.ru

Any questions?