datacenter as a service · 2018. 6. 26. · “crisis management” “managed services -...

DatacenterasaserviceAninfrastructureforasecurecyberworld

2

Agenda

• WhatisaDatacenter?• Internationalstandardsfor“bestpractices”onDatacenters• DatacenterTierLevels• Datacenterproject,implementation,operationandmanagement:

Aprovenmethodologyframework• DatacenterasaService- Vulnerabilities,threatsandrisks• DatacentersInfrastructuresArchitecturesinaCyberWorld• Datacentercyberattack– Realcasestudy• Themeaning,theimpactandthefutureofDatacentercertifications• Q&A

SEGURTIPauloBorges

©

90m

3

Who is Paulo Borges?

BSIISO/IEC27001 – AccreditedimplementerBSIISO/IEC27001 – LeadAuditorPECBISO/IEC22301 – LeadAuditorPECBISO/IEC20000 – LeadAuditorPortugueseSecurityAgency – SecurityAuditorandAdvisorUpTimeInstituteATS – DatacenterSpecialist

30yearsofexperience:• InformationSystemsSecurity• RiskManagement• BusinessContinuity• InformationTechnologyarchitectureprojectsandmanagement• IBMPoweretIBMPureSystemsarchitectandauditor

Datacenters:• Strategy,businessplans,technicalsolutions,projectmanagement,

consultantandauditor• Certificationprocessmanagement

SEGURTIPauloBorges

©

4

i-Governancia partnership

http://www.i-gouvernancia.com/

SEGURTIPauloBorges

©

5

What is a Datacenter?SEGURTI

PauloBorges©

6

What is a Datacenter?SEGURTI

PauloBorges©

7

What is a Datacenter CAMPUS?SEGURTI

PauloBorges©

GESTÃODOCAMPUS

CLIMATIZAÇÃO

COMUNICAÇÕESINTERNAS

STAGING

EnergyProduction

CoolingProduction

ExternalCommunications

MMR– ‘MeetmeRoom’IXP– InternetExchangePoint

SafeworkplacesforcustomeremployeesTechnicalparkExternalperimeter

“BusinessContinuity”“Recreationalareas”“CrisisManagement”“ManagedServices- Outsourcing”

FacilitiesOperationsCenter

CoolingDistribution

InternalCommunications

LoadDockingStaging

EnergyDistribution

ITServersRoom(s)StorageRoom(s)

***ITServices***

Technicalpark– Internalperimeter

SecurityOperationsCenter

CampusPhysicalSecurity

8RagingWire - North Virginia - USA

SEGURTIPauloBorges

©What is a Datacenter CAMPUS?

9


PauloBorges©

Telefónica– Alcalá(Madrid)- Spain

10PortugalTelecomCovilhã- Portugal


PauloBorges©

11VivoTamboré – SãoPaulo- Brasil


PauloBorges©

12

International standards for “best practices” on Datacenters projects

SEGURTIPauloBorges

©

13

Datacenter Tier LevelsSEGURTI

PauloBorges©

14

TierLevel

Redundancy DistributionPath ConcurrentMaintenance

Compartmentation ContinuousCooling

I No

Ncapacity

A No No No

II Yes

N+1capacity

A No No No

III Yes

N+1capacity

AandB

(Active-Passive)(Active-Active)

Mandatory No No

IV Yes

Nafter anyfailure

AandB

(Active-Active)

Mandatory Mandatory Mandatory

SEGURTIPauloBorges

©Datacenter Tier Levels - Requirements

15

Datacenter Tier LevelsSEGURTI

PauloBorges©

16

The UpTime Institute

https://uptimeinstitute.com/about-ui

SEGURTIPauloBorges

©

17

Certifications around the world

https://uptimeinstitute.com/TierCertification/

SEGURTIPauloBorges

©

18


Designdocumentscertification:• Englishwrittendocumentsaccordingtoaspecific

UpTimeInstitutemethodology• Notavailableforpublicannouncement

SEGURTIPauloBorges

©

ConstructedFacilitiescertification:• Tobeachievedwithin2yearsofDDcertification• OnsiteauditbyanUpTimeInstituteteam• Availableforpublicannouncement

OperationalSustainability:• Optionalcertification• Focusedonmanagementbasedonprocesses,procedures,

records,metricsandauditplans• CompatiblewithISOmanagementsystemmethodologies

19


https://uptimeinstitute.com/TierCertification/

SEGURTIPauloBorges

©

20

A Datacenter project, implementation, operation and management proven methodology

SEGURTIPauloBorges

©

21

A Datacenter project, implementation, operation and management proven methodology

SEGURTIPauloBorges

©

22

• Privatesystemsonprivateinfrastructures

• Privatesystemsonsharedinfrastructures• HOSTING - OnlyPower,CoolingandCommunications• COLOCATION - Typicallyincludesmanagedservices

• Sharedsystemsonsharedinfrastructures:o PaaS – PlatformasaService

(usedbyvirtualizedsystemsex:WebSitesandWebServices)o SaaS – SoftwareasaService

(usedbyApplicationsServicesex:SAPHANA,ORACLECloud,…)o CloudComputingSofwareServices

§ Private§ Public

Datacenter as a Service: Vulnerabilities, threats and risks

SEGURTIPauloBorges

©

23


SEGURTIPauloBorges

©

24


SEGURTIPauloBorges

©

• Persquaremeter• Perrack(s)• Percomputerroom

• Perreservedpower(Energy+Cooling)• PerkWh(Energy+Cooling)

• Pernetworkport• PerGbps

• Permanagedservicetype• Permanagedserviceconsumption• Permanagedservicecontract

25


SEGURTIPauloBorges

©

• AllDatacentersizes• Complexsystem• Requiresamanagementteam• NoITmanagement

26


SEGURTIPauloBorges

©

27


SEGURTIPauloBorges

©

28


SEGURTIPauloBorges

©

29

SEGURTIPauloBorges

©

DCIM

BMS


Poorintegrationincreasesrisk!

EnergyCooling

FireSecurityPhysicalSecurity

LightingAutomation

Communications

ICTCapacityPlanningICTTrendHistoryICTMigrationScenarios

30

Datacenters Infrastructure Architecturesin a Cyber World

SEGURTIPauloBorges

©

31

Datacenters Infrastructure Architecturesin a Cyber World

SEGURTIPauloBorges

©

32


SEGURTIPauloBorges

©

ICTExposuresICTVulnerabilitiesICTSecurityIssuesIPNetworksdesignRemoteAccess

IPnetworksintegrationIndustrialNetworksdesignNetworkProtocolsEventsandalarmsMaintenanceManagementIncidentManagement

33


SEGURTIPauloBorges

©

Vulnerability Threat RiskIdentification

Best practices

Typicaluse ofcommonIPnetworksandITservers

Common CyberSecurityattacksmaycompromisetheBMSand/ortheDCIM

• Dataloss• Dataintegrity failures• Remoteaccesstoautomation

servers• Remotecontroloffield

equipment(generators,chillers,electricalpanels,etc…)

Networkand ITsystemsphysicalandlogicalsegregationdedicatedtoBMSand/orDCIM

Useof “cleartext”IPcommunications

BMSand/orDCIMDDOSattacks.

Access tonetworktopologyschemas

• Remote accesstomanagementsoftware

• Stackoverflows• Falsealarms

Useof DigitalCertificatesforAuthenticationandEncryption

Useof SNMPtraffic ”Man inthemiddle”CyberSecurityattackstospecificequipment

Remotecontroloffieldequipment(UPS,PDU,ATS,etc...)

UseofSNMPV3withencryption

Lossofchronologicalsequenceofevents

Humanerror • Bad operationsdecisions• Incorrectorimpossible

diagnosis• Chainofincidents

UseofNTP StratumsystemwithUTCexternaltimestampreference

34

Cyber Security for Datacenter

GESTÃODOCAMPUS

CLIMATIZAÇÃO


STAGING

EnergyProduction

CoolingProduction



SafeworkplacesforcustomeremployeesTechnicalparkExternalperimeter

“BusinessContinuity”“Recreationalareas”“CrisisManagement”“ManagedServices- Outsourcing”

FacilitiesOperationsCenter

CoolingDistribution


LoadDockingStaging

EnergyDistribution


***ITServices***

Technicalpark– Internalperimeter

SecurityOperationsCenter

CampusPhysicalSecurity

SEGURTIPauloBorges

©

36


CLIMATIZAÇÃO


EnergyProduction

CoolingProduction



Safeworkplacesforcustomeremployees

CoolingDistribution


EnergyDistribution


***ITServices***

SEGURTIPauloBorges

©

ExternalAccess

37


üMMT(MeetmeRoom)andIXP(InternetExchangePoint)üBMSandDCIM– DatabasesandmanagementfunctionsüMMS(MaintenanceManagement)– Accesstoinventoryitemsü IMS(IncidentManagent System)– Accesstoclientdataü EnergyandCoolingSystems– Remotepowerdownorchangeof“setpoints”

ü SecuritySystems– ToavoidthecollectionofevidencesüDatacenter imageandcredibility

SEGURTIPauloBorges

©

39

1. Excessive exposure information :A manufacturer announced the installation of automation equipment on thisDatacenter with excessive information, including models, on theirs web site

2. The default IP addresses are posted on the equipment manual and were not modified3. The security system allowed Internet exposure of the automation network4. Using a "Man in the middle" attack to the vendor’s office, the cyber attackers accessed

the automation network (a specific segment with more than 800 temperature sensors)5. The .Set Point. of the temperature alarms were raised on the CRAC units6. Two hours after the servers automatically started to shutdown due to over temperature7. Five hours after the Datacenter management announced a service breakdown on the

international television channels

Datacenter cyber attack – Case study #1SEGURTI

PauloBorges©

40

1. Large Datacenter with all the common ICT security controls (more than 800 racks)2. An USB pen drive was found on the service stairs leading to the Operations Center3. The USB pen drive was plugged “to see what was inside”4. Without operations control, a malware was installed in the DCIM using SNMP protocol5. 2 hours after, all the energy grid topology was recognized by the malware6. 6 hours after, half of the generators started to shutdown7. The power inverter to external energy source was not responding8. The UPS systems were reconfigured to retain 50% of autonomy, meaning that they only

allowed 30m of autonomy for critical loads9. One of the UPS systems collapsed after 10 minutes10. After 8 hours all the Datacenter had no energy sources available11. It took 3 days to recover the Energy Grid to normal mode

Datacenter cyber attack – Case study #2SEGURTI

PauloBorges©

41

1. Define an Integrated Security System, possibly based on an ISO 27001 ISMS2. Identify risk levels on the Datacenters infrastructure design, possibly based on the ISO 31000

methodology3. Identify security controls for the Datacenter infrastructure based on a Risk Treatment Plan4. Check for technical solutions appropriated for the security controls implementation5. Create and manage a Business Continuity plan for the Datacenter infrastructure services, possibly

based on the ISO 223016. Create and manage an Incident Management System, possibly based on the ISO 20000 incident

management process7. Systematically train the team on such processes and security procedures8. Manage the Datacenter infrastructure according to the ”The UpTime Institute” Operational

Sustainability methodology9. Implement and systematically audit a BMS secure topology10. Have a continuous improvement security genetics properly spread within your team

Datacenter cyber attack – Best practicesSEGURTI

PauloBorges©

datacenter as a service · 2018. 6. 26. · “crisis management” “managed services -...

Documents