“internet2presentaon”€¦ · project(route(total pennren fiber route is estimated at 1,613...
TRANSCRIPT
![Page 1: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/1.jpg)
“Internet2 Presenta,on” Brian Stengel, Director of Opera-ons, Kinber Peter J. Heverin, Project Manager, Kinber
Mike Carey, Network Engineer, Kinber Jon Paul Herron, Director of Engineering, GlobalNOC
Luke Fowler, Senior Manager, Systems Engineering, GlobalNOC Marianne Chitwood, Director of Opera-ons, GlobalNOC
September 20th, 2012
![Page 2: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/2.jpg)
What is PennREN
• Capital Budget - $128,958,031 – Federal Stimulus Funds - $99,660,678 – Matching Funds - $29,297,353
• Outside Plant Infrastructure Constructed for PennREN – 48 Stands of NZD Fiber optic Cable – 1700± Route Miles – Outsourced Fiber Maintenance
• 13 Optical Regeneration Service Nodes • 56 Service Distribution Access Nodes
a high-speed, state-wide, research & education serving healthcare, ���K-20 and the public good
![Page 3: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/3.jpg)
Project Route
Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following;
• 1,086 miles of new aerial construction. • 486 miles of aerial overlash. • 14 miles of new underground construction. • 27 miles of leased underground conduit.
Network Backbone Engineering is 95% complete overall.
3
![Page 4: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/4.jpg)
Project Route
4
![Page 5: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/5.jpg)
PennREN Loca,ons Service Nodes
Lehigh University
401 N. Broad
Penn State Hershey Medical Center
Indiana University of Pennsylvania
Allegheny Center Mall
Slippery Rock University of Pennsylvania
Penn State Erie (Behrend)
Clarion University of Pennsylvania
Penn State Dubois
Penn State – State College
Windstream
Bucknell University
University of Scranton
Amp Nodes East Stroudsburg University of Pa
Kutztown University of Pa
West Chester University of Pa
Millersville University of Pa
Shippensburg University of Pa
PiQ Johnstown
Community College of Beaver Co
Allegheny College
PiQ Titusville
Penn State Hazelton
Access Nodes 46 Loca,ons on PennREN fiber
![Page 6: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/6.jpg)
Project Teams
• Hardware procurement • Design • Integra,on • Project Management • Professional Services
• Plan/Design • Op,cal engineering • Field services • Professional services
• Staging • Shipping • Configura,on • Tes,ng • Turn-‐up • Field Services
• Plan/Design • Consul,ng • Engineering • Support • Training
• Fiber OSP • Building entrances • Cabinets, Installa,on • Splicing • Tes,ng • Last mile solu,ons
• Earthlink sourcing for OOB MPLS Network
• Last mile solu,ons
• Service Desk, Tier 1, Tier 2 • Systems and Network Tools • Network Engineering • Net, Perf, Change Management • Opera,ons support
![Page 7: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/7.jpg)
Node Installa,on Status
Segment in Produc,on
Equipment Installed
Under Construc,on
![Page 8: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/8.jpg)
Co-‐Loca,on at Member Sites
Construc,on Diagrams
![Page 9: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/9.jpg)
Op,cal Core/DWDM System
![Page 10: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/10.jpg)
Packet Core – MPLS
![Page 11: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/11.jpg)
Plan – Build – Integrate
![Page 12: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/12.jpg)
Peering Points (Ini,al)
Internet
R&E
R&E
Internet
![Page 13: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/13.jpg)
Network Management
External Networks
External Networks
Out-‐of-‐band access
In-‐band management
Performance Measurement Servers at every Service Node
PS1 PS2 PS3
DB
![Page 14: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/14.jpg)
Opera,ons -‐ Support GRNOC provides to KINBER: • Service Desk – 24x7x365 call center support, ,cket management, technical
support coordina,on, and workflow support • Network Engineering – Expert network engineers work with the Service Desk
to ensure fast problem resolu,on, provisioning, and strategic engineering and planning
• Sofware and Systems – Provides support through a fully integrated system of network management, measurement, and visualiza,on tools
The GlobalNOC at Indiana University provides carrier-‐grade opera,ons, tools, and network exper,se while placing a singular focus on the unique requirements of the research and educa,on (R&E) Community GRNOC supports 20+ R&E networks across the country
![Page 15: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/15.jpg)
Opera,ons -‐ Maintenance
• Warranty • Hardware replacement
(pre-‐ship) • Sofware maintenance
• Warranty • Hardware
replacement (pre-‐ship)
• Sofware maintenance
• Fiber Maintenance • Emergency Restoral • Rou,ne
Maintenance • OSP Records
• Host IT/Site teams provide remote eyes and hands support upon request
• KINBER Network Engineers
![Page 16: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/16.jpg)
Founda,on for Services
West DWDM Ring
East DWDM Ring
MPLS PE Switches
ASBR Router
Mul,-‐Degree ROADMs
External Networks
External Networks
3rd Party Providers
![Page 17: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/17.jpg)
On-‐Net: Delivery
Internet
R&E R&E
Internet
Access Node
Access Node
Access Node
Access Node
Access Nodes
![Page 18: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/18.jpg)
Off-‐Net: Delivery
Internet
R&E R&E
Internet
Member Off-‐Net
First/Last Mile Commercial
Co-‐Lo
CX Carrier
Private
NNI
![Page 19: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/19.jpg)
PennREN
EPC – Ethernet Port Connec,on Service
Customer Router
Service Node: PE Switch
Customer Subscribes to 1/10GE Ethernet Port(s)
R&E networks
VPLS (pt – pt, mul,point)
KMEX
Commodity Internet
![Page 20: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/20.jpg)
Services • Services available to a customer with an EPC
Member A
Member A Member C
Member B
![Page 21: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/21.jpg)
Services • Community-‐wide, distributed service for members to
exchange traffic across a common network. Similar to an Internet Exchange
• Member-‐to-‐Member peering, ad-‐hoc R&E ac,vi,es • Best-‐effort traffic exchange within the community
• Transit service to major R&E networks such as Internet 2, ESNet, NLR…
• Provided by KINBER affiliates
• Virtual private networks with commiYed bandwidth can be established using VPLS instances in the PennREN network
• VPWS – Virtual Private Wire Service – Point to Point • VPLS – Virtual Private LAN Service -‐ Mul,point
• Access to commodity Internet service is available over the PennREN network
![Page 22: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/22.jpg)
Op,cal – Wave/Lambda Services
• Op,cal Waves 10G can be provisioned across the network Member A Member A
Member A Member B Member B
![Page 23: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/23.jpg)
• PerfSONAR Measurement Archives allow exchange of data with other network operators
• Regularly scheduled tes,ng across the backbone
• User-‐ini,ated tes,ng for applica,ons such as problem diagnosis
• Mul,ple rou,ng tables allow us to support both 1G and 10G testpoints on a single host
Performance Measurement
![Page 24: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/24.jpg)
Performance Measurement
Deploying performance measurement servers at each service node
3 servers per site:
– Active throughput measurement (1G and 10G) – Active latency measurement – local data collection / ad hoc performance measurement Specific performance measurement tools include:
BWCTL OWAMP MaDDash
24
![Page 25: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/25.jpg)
Network Measurement
• GlobalNOC tool-‐set updated to support all PennREN devices, including: – SNAPP – High-‐resolu,on SNMP-‐based network u,liza,on data
– LLAMA – DWDM layer performance measurement data
– Central storage of other passive data like syslog, configura,on, flow, etc. for regular and ad-‐hoc processing & analysis.
![Page 26: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/26.jpg)
PennREN Service Desk
• GlobalNOC Specialized Support Technician • Footprints PennREN project created – integrated to
TickMon, Opera,ons Calendars, Trouble Tickets • Telephone number for PennREN customers integrated
into shared GlobalNOC phone queue • Email established [email protected] to receive customer
inquiries and or communica,ons from vendors….this is monitored 24x7x365
![Page 27: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/27.jpg)
PennREN Service Desk
• Network and Member impact guidelines defined
• Web form for customers/vendors to submit trouble ,ckets into Footprints
• Change management process/form created • Internal documenta,on developed/published for staff training and reference
![Page 28: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/28.jpg)
PennREN Service Desk
• Pro-‐ac,ve network monitoring • Support for scheduled maintenances and changes
• Vendor coordina,on • Customer install process • Repor,ng • Security • Tools and Communica,ons
![Page 29: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/29.jpg)
Weighing network needs
Reach
Time
Cost
Capability
![Page 30: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/30.jpg)
A new network is more than a construc,on project
• It’s a system: – Goals – Infrastructure – People – Services – Opera,ons – Business
• Make decisions and plans early (also, there will be more decisions than you think, so watch for boQlenecks)
• The work on all of these areas starts right away
![Page 31: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/31.jpg)
Service Defini,on
• Unrealis,c to imagine services will be completely defined from the beginning of the design
• But, enough understanding is needed to guide the design and build plan
• Early “anchor” users help a lot!
![Page 32: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/32.jpg)
Communica,ons
• Communica,on needs: – to be high-‐bandwidth – Early is good – Changing staff can be disrup,ve, do it carefully. – Informal is good – Mul,-‐channel is good – reliable technologies are good – Face to face is good
![Page 33: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/33.jpg)
Documenta,on
• Documenta,on: – Have a place for documents – Keep the place for documents clean – Keep the purpose of each document clear/dis,nct
![Page 34: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/34.jpg)
Opera,ons
• Opera,ons Preparedness: – Start early, there are a LOT of things to think about, especially:
• Turn-‐up/acceptance process • Expecta,ons for facili,es • How and where to keep network data
![Page 35: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/35.jpg)
Build-‐outs make for lots of informa,on
• Need to be ready to put it somewhere where it’ll be usable later
• Data entry is cheap • Entering/documen,ng is best when the informa,on and its context are fresh
![Page 36: “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following; • 1,086 miles of new aerial construction](https://reader035.vdocuments.us/reader035/viewer/2022070801/5f02ada37e708231d405768b/html5/thumbnails/36.jpg)
The lowly management network
• Some,mes doesn’t get enough aQen,on • It can get crazy complicated or crazy expensive
• NOTE: in an SDN world, this becomes even MORE important!