Parallel session E:Campus networking

Chair: Rob Evans

SPONSORED BY

Please switch your mobile phones to silent

19:30

No fire alarms scheduled. In the event of an alarm, please follow directions of NCC staff

Dinner (now full)Entrance via Goldsmith Street

16:30 - 17:30

Birds of a feather sessions

15:20 - 16:00

Lightning talks

Campus disasters –Are you ready

for Storm Desmond 2, too?Richard du Feu, Lancaster University

03/05/2023

About Lancaster University

»Campus based»On top of a hill (immune

from flooding?)»Fibre rich»Up to 50% of power

generated on site (CHP, 2.3 MW wind turbine)

»7,000 students living on campus

Campus disasters - Are you ready for Storm Desmond 2, too?

Campus from the air (with power) – Chad Conway

03/05/2023

Overview

»Emergency planning»Storm Desmond› Situation› Initial response› Short term efforts (until power restoration)› Post-incident

»Longer term developments

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023Campus disasters - Are you ready for Storm Desmond 2, too?

Planning – Project Hydra

»Two Emergency planning exercises over last decade

»Not always taken seriously by many on the ground

»All exercises ended after 12 hours

»Weaknesses identified and some rectification steps taken

03/05/2023

Planning – Testing backup power

»Regular data centre generator testing

»Annual power failure tests to data centre backup power systems

»UPS calibration runs aimed for every 6 months, in reality annually around Christmas

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

5th December 2015 - Storm Desmond

»Up to 340mm of rain in 24 hours in the Lune catchment

»1600 cubic metres per second (Olympic swimming pool in 1.5 seconds)

»Substation for Lancaster flooded

»Flood defences designed for 1 in 100 year flood

»61,000 homes without power for c48 hours

»Lancaster University without power for 4 days

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Immediate effects

»Power went off on Saturday 5th December at 22:45pm to the University

»Students generally left in rooms until morning as if they’re asleep it’s not a problem

»Water ran out (it’s pumped, but students ok with that)

»Sewage stopped flowing (it’s pumped, but students OK with that!)

»UPS batteries went flat, WiFi stopped…

21st century Maslow

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Initial response

»Emergency management team (EMT) called

»Space for EMT had networking generator backed up, no power to rest of building!

»Decant space for students generally on multimode fibre being fed from distribution switches on UPSs – No networking in emergency spaceCampus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Short term

»EMT not planned for emergencies going on longer than 12 hours

»Mobile cell batteries ran out

»Campus radio repeater needed moving due to battery issues

»Zero communication available away from a small number of buildings with power and POTS phones

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Communications

»Limited WiFi»Very limited 3G»No (useful) radio

»How to coordinate people to tackle problems as they become apparent?› IM is the way forward however

which one - Skype, Jabber, facebook messenger?

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Magic yellow boxes of internet

»For various events we have Wireless APs mounted in waterproof cases

»Ideal for getting wireless outside buildings with power and networking

»Point-to-point radio link allow boxes to be generator powered

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Power restoration to data centres

»Data centres are generator backed up

»No loss of service on loss of grid power (the plan worked!)

»Transition from generator back to mains appeared to go OK… until the UPS went flat

»Sticky switch that failed in tests but OK second time failedCampus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Power restoration

»UPSs»Type B breakers»Client authentication»Failures at power on»BMS/Air conditioning»Leaks

How well does your network restore?

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Other things to be aware of

»In a crisis everyone is busy – help them out, particularly the porters and security team

»Maglocks!»ACLs and your NOC»Single laptop screen

makes much of the response difficult

»Overwhelming of monitoring

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Summary

»Test generators and UPSs

»Have emergency exercises› Limit them to likely

scenarios»Put in enough SM fibre

› 1 core per 12 data lines?»Control costs»NOC generator backed

up.»In a crisis be prepared to

be flexible

»If you need to fix something, go incognito

»Make sure your decant space is known and low on SPOFs

»Post incident if you offer a solution it will be taken up…

»The window for money is very small (days)

»It’s all about the WiFis.Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Are we ready for Desmond 2?

»Most access switches connected directly to generator backed up locations

»Increased fibre count in specification

»Replaced Campus radio system to be digital and more resilient

»Are we ready? No. Will it be less painful? Yes.

Campus disasters - Are you ready for Storm Desmond 2, too?

03/05/2023

Thanks for listening;Any questions?

Campus disasters - Are you ready for Storm Desmond 2, too?

jisc.ac.uk

03/05/2023

contact

Richard du FeuLancaster Universityr.dufeu@lancaster.ac.uk

Campus disasters - Are you ready for Storm Desmond 2, too?

Adventures in wireless, Or how to throw coffee

accuratelyRenyk DeVandre, Coventry University

WHY IS MANAGING WI-FI IMPORTANT:MISSION CRITICAL

•2015 HE survey mission critical 72.5%•Weapons Grade Wi-Fi•Standalone Service not an afterthought to your wired

GROWTH •Cablecom found 92% of students with 2 devices as high 6 •SAP - At the end of 2013 there were more mobile devices than people•IPASS – 888% wifi growth worldwide since 2013 Wifi growth

EXPECTATIONS

•As speeds increase so will expectations•Wi-Fi is the next utility - PERVASIVE•Five 9’s Availability•Many devices don’t come with an Ethernet port now!

STRAGEGIC RESOURCE

•NSS scores – It is important to students•Young people (that’s your students) prefer the internet to daylight hot water and sleep!•Critical to enable mobile working - BYOD

SOUND FAMILIAR:Inherited a

Wi-Fi Network

•No Design methodology•No documentation•How do you solve these issues…..

Implementing a new Wi-Fi network

•No standards to work from•No usage/requirements

The all wireless network

•Why do we need wires it’s the 21st for heavens sake•50 users high speed transfers please

Gigabit Wi-Fi

•Why can’t I transfer at gigabit speeds?

YOU WILL ENCOUNTER SOME OF THESE:SSID

OVER

HEAD

BANDWIDTH STEER

INGWIRED

CAMERAS,

BMS,

ACCESS CONTROL?

GOOD

COVERAGE

PERFORMANCE

IS AWFULI DON’T UNDERSTAND WHAT

YOUR TELLING

ME

JUST ADD MORE AP’S FOR BETTER

WIRELESS

KEY THEME 1 – TALK A LANGUAGE PEOPLE CAN UNDERSTANDWhy is Wi-Fi so

difficult to communicate?

•Lack of understanding•Common language•More complex than wired

Vendor Truth •What's on the box is rarely in the tins•Stretching the truth•Get your vendor/partner in to present – make the earn your money

Understanding

•Ubiquitous Wi-Fi is not broadband•802.11ac does not mean Gigabit networking

Training •You and your engineers need to understand the technology to explain it.•When budgets are tight training goes first – resist•CWNA – CWNP – Vendor Neutral

HOW TO THROW COFFEE

ACCURATELY•Try throwing coffee – landing in the same place•Now try with people moving in front•Now make sure everyone gets enough to drink from one cup

KEY THEME 2 - UNDERSTAND YOUR ENVIRONMENT

You can’t control what you don’t understand

• Internal environment• External environment what's

around you?Visual• Engage with stakeholders (MAP

THEM)• Challenge stakeholders• Questionnaires

Stakeholders

• Patterns inform your design• Monitoring/Management tools• VoIP, Video, RTLS Usage• Other institutions• JISC• Local Council – Opportunities?

Peer Support

• Direct engagement with the vendor• Best Practice documentation• Architecture & RoadmapsVendor• Engage with a specialist designer

surveyor• Full Surveys – NOT JUST COVERAGEExperts

KEY THEME 3 – SURVEY\DESIGNNATURE V’S NURTURE

StakeholdersAcademicBusinessStudentExternal

Business

Partner

Designer

KNOWLEDGE •Informs your design•Informs your vendor selection•Comes from understanding your environment

Architecture •It’s not just your front end design•Get our architecture right – 14 controllers•Design for VoIP, Video, RTLS

Wireless SLA •Do you have one?•Survey/Design guide – Do you have one?•What's the relationship?

Design for capacity

•Green means nothing•In the absence of numbers assume high usage

Function over form

•If the stakeholder overrides the design – get it in writing •If the project cuts corners – get it in writing

Set your deliverables

•Can form a KPI•A benchmark for success•Demonstrates ROI

DESIGN MEETING

KEY THEME 4 - VENDOR SELECTION:Informed Design

•Any vendor will work (almost)•As long as it’s standards based•Ours is better - theirs doesn’t work “myth”

Requirements/InformationGathering

•Your vendor informs your design•Critical to leverage all features•Road mapping

What do you want from wireless?

•Features– which vendor says they can deliver?•Compare features – you didn’t know you wanted that•Don’t forget the “vision” tools, management, integration!

The real story •Look for non vendor use cases•Peers institutions will give you the true story•One visit set up by a vendor I was told “switching is good wi-f don’t touch it!”

KEY THEME 5 - FINANCIALSWired V’sWireless

•25 yr. investment•Full control•Faster

Project Sign Off •Cutting corners? - Put it in writing not a warranted design•Specify the long term benefits

Training & Staffing •You have an X Million service you need X to leverage it.•None of this happens without knowledge•Making the case for a specialist

Savings •Good Designs save money 62 AP’s v’s 144 AP’s•Average wasted AP’s = 20% in older installations as high as 32% !!!!

QUESTIONS?

Renyk de’VandreNetwork Services ManagerCoventry Universityab7597@Coventry.ac.uk

jisc.ac.uk

Thank you

Renyk de’VandreNetwork Services ManagerCoventry Universityab7597@Coventry.ac.uk

03/05/2023Adventures in wireless, Or how to throw coffee accurately

LSE’s campus refresh – not just about the tin

Ed Spick, Matt Bernstein, LSE

LSE’s Campus Refresh:It’s not just about the tinNetworkshop 45, 12/04/2017Campus NetworkingEd SpickLSE Network MangerMatt BernsteinLSE Senior Network Architect

Contents• About LSE and its Campus Network

(Ed)• Reshaping team and environment (Ed)• Refreshing tin and topology (Matt)• Looking to the future (Matt)

About

• est. in 1895“for the betterment of society”

• 200+ public events per year• 18 Nobel Laureates

amongst its Alumni• 2nd in the world (QS 2017)

for the Social Sciences• 10,800 students

• 5,000 undergraduate• 5,800 graduate

• 3,300 staff

LSE’s Campus• 40 buildings in Central London• Students from over 150 countries• Major Capital Development projects• 10 Halls of Residence (out of scope)

Campus Network• 31,000m of fibre optic cable in

dedicated ducting

• 90 comms rooms

• 685 network devices

• 37,000 network outlets

• 650 wireless access points

• 3,500 telephone handsets

Original Team Structure

• Flat team structures

• Permanent, fixed term, and contractor

• Separate Data and Telecoms Teams

LSE Campus Refresh Project Phases

New Team Structure

Challenges of Team Growth

• Truckman’s stages cycling• Growing out of small team mindset

• Having a Mythical Man-Month problem?• Brooks’ Law of ramp-up time or “operational drag”

Forming

Storming

Norming

Performing?

Environmental Challenges• Shallow comms racks• Poor cabinet layout• Congested• Overheating

The same facility after:• Extensive planning• 8 weekends’ migration• Cabinet replacement• Estates coordination• Air conditioning

Changes in campus topology

• November 2011• 4½ core locations• Some resilience• Laser / WiFi links• Lots of SPoFs• DCs on campus network• Little evidence of design

CS-R

CS-V6

1Gbps

10+1G

FTPVPN

NH, GH, LK Halls

BW, SW Halls

Telecoms

Peacock

K-2

Lionel Robbins

MCRServices

SCRServices

DCR Services

LSE Research LabSTICERD/CASE 5th flr

CEP 4th flrServices

Library Services

LMN-1ULCC

LMN-2KCL

1Gbps

10G

PacketShaper

HH, BS, RB, CS, PF Halls

Old Building

St Clements Building

50 LIF

Sheffield St

Kings/Lincoln Chambers

Cowdray House

Lakatos Building

Clement House

Tower 3

Tower 2

Tower 1

Clare Mkt

East Bldg

Columbia House

ConnaughtHouse

20 Kingsway

1+1 Gbps

1 spare unused cable

S-R

S-V

A-V

SPoF

edge

edge

802.11a

1+1 Gbps

1+1 Gbps

Thus switch5 links

2 links

Tower 2

8 subsidiary comms rooms F-LG to F-7

LTC

14 subsidiary comms rooms

TC2

TC2Services

NABF-B

10G

10G

F-7

edge

New Court

CS-A

CS-TC2

CS-LTC

CS-V2

1+1 Gbps

CS-S

100Mbps

10G

10G

K-B

L-G

10+1G

Sardinia House

Maths

Finance

FMG

100Mbps

10G

10G

SPoF

SPoF

SPoF

SPoF

32LIF(in test)

AldwychHouse

Changes in campus topology

• April 2017• Two core locations• Air-gapped DCs• Regular pattern

emerging• 10Gb/s nearly

everywhere - VSL -

- VSL -

- Nexus FabricPath -

- Chassis Cluster Link -

10Gb/s to Telecity Powergate

NH, GH, LK HallsBW, SW Halls

10Gb/s to Imperial College

HH, BS, RB, CS, PF Halls

MCRServices

Core-2 (STC)

SCRServices

95A

1KW

PAR

Security Lodge

KSW

PEA

50L

NAB

32L

LRB

LCH

LAK

KGS

SHF

SARSAW

CLM

COW

OLD

ALD

STC

TW2

CON

COL

QUE

Core-2 (TW2)

Core-3 (TW2)

Core-3 (STC)

Campus Core• 6509 VS4O pair retained

• for WiSMs + L2 buildings

• C6807 Sup6T VS4O new core• pure L3• largest attached network is a /31• Lots of interfaces; can connect

30 buildings at 4 x 10GbE• 2 new core locations

Firewall &Janet links

2011

CoreCS-S

CoreCS-AS-A Trunk

CoreCS-V2V-A Trunk

S-V Trunk

ISG2000-1 ISG2000-2

VPN/DMZ VPN/DMZ

Halls

HA

Lonman2Cisco 7206

DMZ, VPN servers

Maths/FMG

Halls

Halls

Eduroam

LMN (KCL)

DMZ, VPN servers

Halls

Media convertor

HA

Media convertor

Maths, Finance and FMG systems outside firewall

LMN (ULCC)

Packeteer

Lonman1Cisco 7206

Lonman1/Lonman2 BGP link

Halls

Halls

Halls

Eduroam

Maths/FMG, VPN/DMZ, Eduroam

HA

Lonman1/Lonman2 BGP link

HA

Telecoms firewall bypass

Telecoms firewall bypass

22

Maths/FMG, VPN/DMZ, HA, Eduroam,

Halls, BGP

Maths/FMG, VPN/DMZ, Eduroam

Maths/FMG, VPN/DMZ, HA, Eduroam,

Halls, BGP

Not currently working Core

CS-V6

Eduroam

2017

xe-1/0/4146.97.139.130/302001:630:0:9001::37e/126

xe-1/0/4146.97.139.126/30

2001:630:0:9001::382/126

Janet6 (Telecity

Powergate)MX240 (river)

158.143.223.201 (internet)2001:630:9:f223::1 (internet.inet6)

158.143.223.195 (edge)172.27.221.246 (RSVP/OSPF NHRP)

172.27.221.{253,254} (msmic)

MX240 (press)158.143.223.202 (internet)

2001:630:9:f223::2 (internet.inet6)158.143.223.193 (edge)

172.27.221.245 (RSVP/OSPF NHRP)172.27.221.{251,252} (msmic)

Janet6 (Imperial College)

xe-1/0/0

xe-1/0/1

xe-1/0/0

xe-1/0/1

xe-1/0/2 xe-1/0/2 xe-1/0/3xe-1/0/3

xe-1

/0/8

xe-1

/0/9

xe-1/0/9 xe-1/0/8

irb.0158.143.221.10/31

irb.0158.143.221.8/31 reth0.0

158.143.221.9/31reth1.0158.143.221.11/31

xe-2/2/7 xe-2/2/1

xe-2/2/0

xe-2/2/6

xe-5/2/0

xe-5/2/6

xe-2/2/4xe-2/2/2 xe-5/2/4 xe-5/2/2

xe-2

/2/3 xe-2/2/5 xe-5/2/3

xe-5

/2/5

xe-5/2/7xe-5/2/1

reth2 reth2

reth3.481: 172.27.2.90 (Sites)reth3.484: 158.143.221.17

Vlan481: 172.27.2.91Vlan484: 158.143.221.16Te1/5/1 Te1/2/1 Te2/5/1 Te2/2/1

ae0.420: 158.143.221.2/31(edge)ae0.421: 2001:630:9:f223::1:5/126 (edge.inet6)

ae0.422: 172.27.221.16/31 (RSVP/OSPF NHRP)ae0.423: 158.143.221.0/31 (internet)ae0.424: 2001:630:9:f223::1:1/126 (internet.inet6)

ae0.420: 158.143.221.3/31(edge)ae0.421: 2001:630:9:f223::1:6/126 (edge.inet6)

ae0.422: 172.27.221.17/31 (RSVP/OSPF NHRP)ae0.423: 158.143.221.1/31 (internet)ae0.424: 2001:630:9:f223::1:2/126 (internet.inet6)

Po203158.143.221.18

Po3158.143.221.19

Te2/2/3Te2/5/3Te1/5/3

Te1/2/3

e1/5

e1/3

e1/4

e1/1 e1/2 e1/13

e1/2

e1/2

e1/2

e1/1

e1/1

e1/1

e1/10 e1/10

RLABvia NAB

RLABvia LRB

e1/6

e1/6 e1/6

e1/6

e1/5

e1/16 e1/16

e1/3

e1/3

e1/4

e1/4

e1/3

e1/5e1/5

e1/4

e1/13

e1/30

Gi1/0/4Gi1/1/1

Te1/1/1Te1/1/3

Te1/1/4

e1/1

6

e1/1

6

Te2/7/19 Te1/7/19

Te2/8/19 Te1/8/19

Te2/7/18Te2/8/18 Te1/7/18

Te1/8/18

ISR4451-X(The Cloud)

NX5548UP (N5K-3)172.27.208.23

NX5548UP (N5K-4)172.27.208.24

C6509E(core 2; sw 1)158.143.223.203

FEX 151FEX 150FEX 100 FEX 140 FEX 141 FEX 101

NX5548UP (N5K-2)172.27.119.12

C6509E(core 2; sw 2)158.143.223.203

FEX 150 FEX 151 FEX 160 FEX 161

NX5548UP (N5K-1)172.27.119.11

e1/7

e1/8

e1/1

1 e1/12

e1/7

e1/8

e1/1

1

e1/1

2

e1/7

e1/2

3

e1/1

3 e1/14

e1/7e1/23e1/14e1/13

43V483 V1020158.143.223.125

MAG6610SSLVPN1

MAG6610SSLVPN2

ports: 1: BS 3: PF4: RB 5: CS 6: HH

9: SW 10: BW

ports: Gi1/0/10: GHGi1/0/11: LKTe1/1/2: NH

Halls172.27.0.240

Halls172.27.0.241

4 4

SRX5400 (zuul; node 0)158.143.223.197 (edge, inc VPNs)

158.143.223.200 (master)

SRX5400 (zuul; node 1)158.143.223.197 (edge, inc VPNs)

158.143.223.200 (master)

C6807XL (core 3; switch 1)158.143.223.199

C6807XL (core 3; switch 2)158.143.223.199

19 20

Consolidating the Access Layer• C3850 does not physically fit into some of our current facilities• L2 but IP Base licensing for Netflow, TrustSec on every access port• “multi-Gig” interfaces (100Mb/1Gb/2.5Gb/5Gb/10Gb)• Programmable ASIC, Cisco’s strategic platform

ReferenceArchitecture(physical)

core

distribution

access

ReferenceArchitecture(logical)

LSE Wireless Uptake

Zoning and NAC• necessary to support multiple

“tenants” on one campus• legal and regulatory compliance• TrustSec driven by ISE• Supports mobility e.g. moving

equipment around Campus• Fundamental driver for our

Business Case

EPOS system

climate change

research data

CCTV etc etc

A Programmable Future?• NETCONF / YANG• Monitoring / Analytics • APIC-EM / Campus

Fabric• Zoning / multi-tenancy

Lessons LearnedChallenges

• External review to make the case for change• Managing “organic growth”

of campus network• Supporting the team as it grows• Coping with legacy environments

Opportunities• Agree regular maintenance windows

with your HEI• Reserve roles for long term planning

and investment• Engage with Campus development projects• Align with new product roadmaps

and reference architectures

Q & A and Credits• Thanks to all of the Network Team• Shameless plug – we’re recruiting – join us!

Thank you

03/05/2023Title of presentation (Insert > Header & Footer > Slide > Footer > Apply to all)