the university of new hampshire interoperabilitylaboratory
TRANSCRIPT
![Page 1: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/1.jpg)
The University of New Hampshire InterOperability Laboratory
(UNH-IOL)
www.iol.unh.edu
Introduction to Broadband Forum WiFi Performance Testing
#UNHIOL#WiFiwebinar
![Page 2: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/2.jpg)
•Who, What, When, Why, Where?•Broadband Forum Test Plan Scope•Implementation of testing
oAvoiding common mistakes•Looking to the future
2
Agenda
![Page 3: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/3.jpg)
•Why performance testing? Why Broadband Forum? Why now?
•WiFi has become the defacto connection for devices in the home and small business
•Making WiFi performance a direct impactor on customer QoE
•Service providers need industry tools to performance WiFi devices
3
Setting the Stage
![Page 4: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/4.jpg)
•Challenge:oService providers must improve the QoE for risk of loosing customers
•Solutions:oProvide customers with well known (tried & tested) solutionsoCooperate with manufacturers to test WiFi equipment before field deployment
§ Repeatable testing to easily set baselines and find deviations caused by software or hardware changes
4
Wireless at the Edge
![Page 5: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/5.jpg)
•Scope:oFocused on testing of performance for AP devices (i.e. equipment provided by the service provider)
•Sections:oRF CapabilitiesoBaseline PerformanceoCoverageoHandling Multiple StationsoStability & Robustness
5
Test Plan Scope
![Page 6: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/6.jpg)
•Broadband Forum test plan includes absolute performance requirements o Relative requirements are pass/fail metrics based on previous measurements for the same DUT
o Absolute requirements are metrics relative to “zero”
•Broadband Forum test plan is unique in this aspect
6
Absolute vs Relative
Baseline
Absolute
Measurement
Relative
![Page 7: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/7.jpg)
•Receiver Sensitivity – Measurement of DUT’s ability to receive weak (small) signalso Impacted by: Chipset, Antenna Design, Internal Noise
7
RF Capabilities
![Page 8: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/8.jpg)
•Maximum Throughput Test – Measure the DUT’s maximum throughput under ideal conditionso Impacted by: Chipset, Antenna Design, System Software, CPU, Internal Noise
•Maximum Connection Test – Verify the DUT supports a minimum number of concurrently connected stations (STA)o Impacted by: Chipset, System Software, CPU
•Airtime Fairness – Verify the DUT fairly allocates “airtime” when multiple stations are actively passing traffico Impacted by: Chipset, System Software, CPU
8
Baseline Performance
![Page 9: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/9.jpg)
•Range Verse Rate – Measure the average throughput supported at different ranges (attenuation) between he DUT and STAo Impacted by: Chipset, Antenna Design, System Software, CPU
•Spatial Consistency – Measure the average throughput as a function of DUT orientation to the STAo Impacted by: Antenna Design
9
Coverage Testing
![Page 10: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/10.jpg)
• Multiple Stations Performance – Measures the throughput between DUT and STA, when other STA are associated to the networko Impacted by: Chipset, Antenna Design, System Software, CPU
• Associations / Dissociations Stability – Verifies “non-impact” on throughput while other stations leave / join the networko Impacted by: Chipset, System Software, CPU
• Downlink MU-MIMO Performance – Measures throughput for systems implementing 802.11ac MU-MIMO for downlink (AP to STA)o Impacted by: Chipset, Antenna Design, System Software, CPU
10
Handling Multiple Stations
![Page 11: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/11.jpg)
•Long Term Stability – Verifies the link is stable over long duration testing (24 hours)o Impacted by: Chipset, System Software
•AP Coexistence – Measures the system performance when other WiFi networks are in operation within the same “air space”o Impacted by: Chipset
11
Stability & Robustness
![Page 12: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/12.jpg)
12
Implementation of Testing
•AP (DUT)•Station(s)•Controllable RF Path
•Clean RF Environment
•Packet Generator
“CleanAir”“DUT” “STA”
PacketGenerator
![Page 13: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/13.jpg)
13
Implementation of Testing (2)
•Isolation ChambersoTurntableoNearfield antennas
•MPE2 – Multi-path emulator•Attenuator(s)•PAL Partner Device(s)•Noise Generator
![Page 14: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/14.jpg)
•RF path is emulated with multi-path fading channel and attenuatorsoDrives repeatability & allows for automation
•Station(s) (STAs) are based on test & measurement equipment. Supporting latest 802.11 standards
•Test traffic can be generated using either iPerf or dedicated test & measurement equipmentoNot yet fully specified in draft test plan
14
Implementation of Testing (3)
![Page 15: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/15.jpg)
0
100
200
300
400
500
600
700
0 1 2 3 4 5 6 7 8 9 1011121314151617181920212223242526272829303132333435363738394041
WiFiThroughput(Mbps)
802.11nUpstream 802.11nDownstream 802.11acUpstream 802.11acDownstream
15
Some Initial Results
0
100
200
300
400
500
600
700
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
Throughp
ut(M
bps)
Attenuation(dB)
WiFiThroughput
802.11nUpstream 802.11nDownstream 802.11acUpstream 802.11acDownstream
![Page 16: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/16.jpg)
• Working with companies around to the world to develop and compare test results: we’ve seen many common mistakes.
• Noisy environmento Avoid any open air testingo Careful with RF connectors (i.e. proper tightness, etc.)
• Calibrated (controlled) RF patho Avoid purely flat paths (i.e. cable and attenuators only)
• Traffic measurement techniqueso Configuration settings can have impact on measured results. Ideally,
final test plan will provide very detailed requirements or iPerf command recipe.
• Dealing with number of spatial streamso Difference(s) between AP and STAo RF path impacts, number of channels, attenuators, etc.
16
Common Pitfalls
![Page 17: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/17.jpg)
17
Projected Timelines
2018 2018
Today
May Jun Jul Aug Sep
BBFQ2MeetingJun11
BBFQ3MeetingSep10
TestPlanPublicPubished
Sep21UNH-IOLpre-testingavailableJul9
UNH-IOLformaltestingavailableAug27
DevelopmentMay1- Jun14
StrawBallotCommentPeriodJun25- Jul23
CommentResolutionsJul23- Aug20
FinalBallotAug27- Sep10
UNH-IOLDevelopingTestingMay1- Aug20
TestPlan“insteadystate”
![Page 18: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/18.jpg)
• UNH-IOL provides fully automated testing
• Testing performed in lab, Durham, NH, USAo Company engineers not required onsite
(but welcome if they would like to attend testing)
o Testing performed by UNH-IOL technicians
• Results owned directly by your company, not published by UNH-IOL or sent to 3rd party without authorization
18
Getting Tested
![Page 19: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/19.jpg)
•Multi-AP Performance (client steering)oStation roaming between multiple APoMAP protocol under development of WFAoAP Uplink is wired or wireless
•Mesh / Hybrid NetworksoLinks between APs provided by 802.11 onlyoFocus on performance through the mesh “uplink”
•QoE Testingo Impacts of Wireless channels on higher layer applications
19
In the Future
![Page 20: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/20.jpg)
20
#UNHIOL#WiFiwebinar
![Page 21: The University of New Hampshire InterOperabilityLaboratory](https://reader031.vdocuments.us/reader031/viewer/2022012103/616a129411a7b741a34e8118/html5/thumbnails/21.jpg)
•Lincoln Lavoieo UNH-IOL Senior Engineer, Broadband Technologieso [email protected]
•UNH-IOL WiFi Testing Services:o https://www.iol.unh.edu/testing/mobile/wifi
•octoScope Test Equipment:o https://www.octoscope.com/
•Broadband Forum Test Plan:o https://issues.broadband-forum.org/browse/CONTRIB-20265(draft, members only)
21
References & Contacts