oleg natalushko. drupal server anatomy. drupalcamp kyiv 2011
DESCRIPTION
Любой полезный ресурс рано или поздно выходит за рамки «shared hosting» тарифов и владельцы ресурса начинают поглядывать в сторону VDS, выделенных серверов и облачных решений. Из нашего доклада Вы узнаете: - стоит искать другого хостера или уже пришло время переезжать на выделенный сервер; - о выборе площадки для аренды или размещению своего сервера; - о выборе ресурсных характеристик сервера, подборе и конфигурации ПО; - что делать дальше, когда сервер установлен. Многие моменты доклада будут украшены реальными примерами из опыта работы компании «IT Patrol inc.»TRANSCRIPT
Drupal Server Anatomy
Presented by IT Patrol inc.
Gold Sponsor ofDrupalCamp Kyiv 2011
Silver Sponsors ofDrupalCamp Kyiv 2011
• Should I look for another hosting company or it is time for a dedicated server• How to choose hosting company for
dedicated server• How to choose dedicated server for your
needs• What next ?
Plan
Possible “Overload” Reasons
• Bots indexing your website• DDOS attack• Habr-effect• Drupal modules with known
performance issue• The problem from theme part of drupal• Your website actually grew up and need
more resources
Bots Access Log Example
*** [11/Apr/2011:03:39:09 +0400] ******* Googlebot/2.1
*** [11/Apr/2011:03:39:10 +0400] ******* bingbot/2.0
*** [11/Apr/2011:03:39:10 +0400] ******* Googlebot/2.1
*** [11/Apr/2011:03:39:10 +0400] ******* YandexBot/3.0
*** [11/Apr/2011:03:39:10 +0400] ******* msnbot/2.0b
*** [11/Apr/2011:03:39:11 +0400] ******* YandexBlogs/0.99
*** [11/Apr/2011:03:39:12 +0400] ******* Yahoo! Slurp
*** [11/Apr/2011:03:39:12 +0400] ******* YandexAntivirus/2.0
Casper Bot Search Alexibot Art-Online asterias BackDoorbot Black.Hole BlackWidow BlowFish botALot BuiltbotTough Bullseye BunnySlippers Cegbfeieh Cheesebot CherryPicker ChinaClaw CopyRightCheck cosmos Crescent Custo DISCo DittoSpyder DownloadsDemon eCatch EirGrabber EmailCollector EmailSiphon EmailWolf EroCrawler ExpresssWebPictures ExtractorPro EyeNetIE FlashGet Foobot FrontPage GetRight GetWeb! Go-Ahead-Got-It Go!Zilla GrabNet Grafula Harvest hloader HMView httplib HTTrack humanlinks ImagesStripper ImagesSucker IndysLibrary InfonaviRobot InterGET Internet Ninja Jennybot JetCar JOC Web Spider Kenjin.Spider Keyword.Density larbin LeechFTP Lexibot libWeb/clsHTTP LinkextractorPro LinkScan/8.1a.Unix LinkWalker lwp-trivial Mass Downloader Mata.Hari Microsoft.URL MIDown tool MIIxpc Mister.PiX Mister PiX moget Mozilla/3.Mozilla/2.01 Mozilla.*NEWT Navroad NearSite NetAnts NetMechanic NetSpider Net Vampire NetZIP NICErsPRO NPbot Octopus Offline.Explorer Offline Explorer Offline Navigator Openfind Pagerabber Papa Foto pavuk pcBrowser Program Shareware 1 ProPowerbot/2.14 ProWebWalker ProWebWalker psbot/0.1 QueryN.Metasearch ReGet RepoMonkey RMA SiteSnagger SlySearch SmartDownload Spankbot spanner Superbot SuperHTTP Surfbot suzuran Szukacz/1.4 tAkeOut Teleport Teleport Pro Telesoft The.Intraformant TheNomad TightTwatbot Titan toCrawl/UrlDispatcher toCrawl/UrlDispatcher True_Robot turingos Turnitinbot/1.5 URLy.Warning VCI VoidEYE WebAuto WebBandit WebCopier WebEMailExtrac.* WebEnhancer WebFetch WebGo IS Web.Image.Collector Web Image Collector WebLeacher WebmasterWorldForumbot WebReaper WebSauger Website eXtractor Website.Quester Website Quester Webster.Pro WebStripper Web Sucker WebWhacker WebZip Wget Widow [Ww]eb[Bb]andit WWW-Collector-E WWWOFFLE Xaldon WebSpider Xenu's Zeus
List of Bots that is NOT important
#block bots
RewriteCond %{HTTP_USER_AGENT} Aport [OR]
RewriteCond %{HTTP_USER_AGENT} Ezooms [OR]
RewriteCond %{HTTP_USER_AGENT} Yahoo [OR]
RewriteCond %{HTTP_USER_AGENT} TwitterFeed [OR]
RewriteCond %{HTTP_USER_AGENT} MJ12bot [OR]
RewriteCond %{HTTP_USER_AGENT} rss [OR]
RewriteCond %{HTTP_USER_AGENT} YandexBot [OR]
RewriteCond %{HTTP_USER_AGENT} bingbot
RewriteRule ^.* - [F,L]
Bots Blocking Rule in .htaccess
DDOS Attack
Habr-effect(Slashdot effect)
Bad Code Example
?
< 1000 visits/day < 5 req/min, < 1000 nodes < 30 req/min, < 1000 nodes, 90% bots
Should I look for other hosting provider?
?
Network infrastructure, speed guarantees, UPS, backup power, diesel generators, Guarantees replacement time, The qualification of the technical support.
Important Parameters of the Data Center
?
CPUMemoryHard driveBandwidth
Checklist for Choosing Didicated Server
SATA IIVelociraptorSASSCSISSD
HDD Types
RAID0RAID1RAID5RAID10
RAID Types
RAID 1 RAID 5
300 GB SAS
300 GB SAS
1 TBSATA II
1 TBSATA II
1 TBSATA II
1 TBSATA II
Architecture of Our HDD Subsystem
We are using
NGINX and Apache interrelation diagram
How “opcode cache” works
“OpCode cacher” performance diagram
High-performance MySQL server
Corporate website: http://it-patrol.ru
Drupal Hosting: http://dh.it-patrol.ru
Contact email: [email protected]
Any questions?