octoscope smallnetbuilder's wireless testbed

Upload: jamie-brown

Post on 02-Jun-2018

214 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    1/14

    Reprinted by octoScope, Inc. www.octoscope.com

    +1.978.222.3114

    2014

    SmallNetBuilder's

    Wireless Testbed V2

    WI-FI BENCHMARK TESTBED

    BY TIM HIGGINS

    http://www.octoscope.com/http://www.octoscope.com/http://www.octoscope.com/
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    2/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 1 www.octoscope.com

    Tel.: +1.978.222.3114

    How We Test: SmallNetBuilder's Wireless Testbed V2MON, 28 JUL 2014 06:00TIM HIGGINS

    Introduction

    After spending the past year or so with ouroctoScope-

    based wireless testbed,I was ready to listen when

    octoScope suggested upgrading.

    The V1 testbed uses octoBox 26" wide chambers, which

    have proved to be a bit too tight, especially as the size of

    routers have grown. The desired 8" distance betweenrouter under test and the chamber antennas has not

    been achieved on more than one occasion.

    Enter the 38" wideoctoBox Stackableanechoic

    chamber. It measures 24" x 38.35" x 31.2" vs. 18" x

    26.35" x 25.2" for the 26" octoBox. The photo below

    shows the new stack, with a new 38" wide octoBox MPE

    (Multi Path Emulator) sandwiched between the upper

    and lower chambers. The key features of the new

    configuration are described in the octoScope videohere.

    The Stack

    The new stack includes

    octoScope's newquadAtten

    programmable attenuator

    module,which replaces the

    multiple Vaunix Lab Brick attenuators in the old setup.

    Dual-shielded cables are also used externally, providing

    better shielding for testenvironmentswith a lot of RF

    activity. The stack sits on a sturdy base with honkin' big

    casters that provide very easy rolling. That's an APC

    Back-UPS 550and TRENDnet TEG-S80gGigabit switch sitting on

    the base in the photo.

    Everything runs off the UPS so that power glitches don't interrupt testing. The TRENDnet switch connects the two

    testbedcomputersto the SNB LAN and each other. RealVNCis used to remotely access the two testbed computers.

    SmallNetBuilder Wireless Testbed V2

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32076-how-we-test-the-technology-behind-smallnetbuilders-new-wireless-testbedhttp://www.smallnetbuilder.com/wireless/wireless-features/32076-how-we-test-the-technology-behind-smallnetbuilders-new-wireless-testbedhttp://www.smallnetbuilder.com/wireless/wireless-features/32076-how-we-test-the-technology-behind-smallnetbuilders-new-wireless-testbedhttp://www.smallnetbuilder.com/wireless/wireless-features/32076-how-we-test-the-technology-behind-smallnetbuilders-new-wireless-testbedhttp://www.octoscope.com/English/Products/octoBox_Stackable/octoBox_Stackable.htmlhttp://www.octoscope.com/English/Products/octoBox_Stackable/octoBox_Stackable.htmlhttp://www.octoscope.com/English/Products/octoBox_Stackable/octoBox_Stackable.htmlhttps://www.youtube.com/watch?v=Raqb_PUF-iYhttps://www.youtube.com/watch?v=Raqb_PUF-iYhttps://www.youtube.com/watch?v=Raqb_PUF-iYhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttp://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_quadAtten_datasheet.pdfhttps://www.youtube.com/watch?v=Raqb_PUF-iYhttp://www.octoscope.com/English/Products/octoBox_Stackable/octoBox_Stackable.htmlhttp://www.smallnetbuilder.com/wireless/wireless-features/32076-how-we-test-the-technology-behind-smallnetbuilders-new-wireless-testbedhttp://www.smallnetbuilder.com/wireless/wireless-features/32076-how-we-test-the-technology-behind-smallnetbuilders-new-wireless-testbedhttp://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    3/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 2 www.octoscope.com

    Tel.: +1.978.222.3114

    Note that the new testbed can handle fourRF paths vs. three in the old testbed. This will let us test the new crop of

    4x4 AC2350 routers, like ASUS' RT-AC87 that has been available for just about a week as I write this. The 38"

    octoBoxes are much roomier inside as the photo below shows.

    SmallNetBuilder Wireless Testbed V2 inside

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/octobox_38_stack_open.jpghttp://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    4/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 3 www.octoscope.com

    Tel.: +1.978.222.3114

    Everything is connected as shown in the diagram below. As is our standard, the new process usesIxia's IxChariotto

    generate traffic and measure throughput.

    octoBox based WLAN Testbed block diagram

    Computers

    The V2 testbed has gotten a computer upgrade, too. The Dell Optiplex 790 Small Form Factor (Core i5-2400 @ 3.1

    GHz) formerly used in the lower chamber now connects to the upper chamber and runs theIxChariotconsole and the

    IxChariot test endpoint for the Device Under Test (DUT). The Test STA computer is also a Dell Optiplex SFF

    machine, but a 9010 model with Core i5 3570 CPU @ 3.4 GHz. After a briefly wrestling with Windows 8 quirks, I

    decided to stay with Windows 7 Pro on both systems.

    Both computers have a TP-LINK TG-3468PCIe Gigabit Ethernet adapter (Realtek RTL8168Bbased) installed to

    provide a second port. This adapter replaces the aging Intel CT's in the V1 testbed that were starting to become

    unreliable. One port connects to the SNB LAN for control, the other to the DUT or reference STA for test data.

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.ixchariot.com/http://www.ixchariot.com/http://www.ixchariot.com/http://www.ixchariot.com/http://www.ixchariot.com/http://www.ixchariot.com/http://www.realtek.com.tw/products/productsView.aspx?Langid=1&PFid=5&Level=5&Conn=4&ProdID=12http://www.realtek.com.tw/products/productsView.aspx?Langid=1&PFid=5&Level=5&Conn=4&ProdID=12http://www.realtek.com.tw/products/productsView.aspx?Langid=1&PFid=5&Level=5&Conn=4&ProdID=12http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/wireless_testbed_v2_block.jpghttp://www.realtek.com.tw/products/productsView.aspx?Langid=1&PFid=5&Level=5&Conn=4&ProdID=12http://www.ixchariot.com/http://www.ixchariot.com/http://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    5/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 4 www.octoscope.com

    Tel.: +1.978.222.3114

    Upper Chamber

    The signals from the device under test (DUT) are picked up by four omnidirectional unity-gain 2.4 / 5 GHz dual-

    band antennas, shown on the right in the photo below. Because the test client is still 3x3, only three antennas are

    currently used. The fourth signal path is sealed with connector caps on both ends. You can see there is now lots of

    room in the cabinet, even with the largest router seen to date, NETGEAR's R8000 Nighthawk X6.

    Some readers pointed out that the all-vertical antenna orientation in the V1 testbed might provide a performance

    edge to routers with vertical antenna polarization. So in the V2 testbed, only one antenna remains vertical; the other

    two are set at 45. The two Ethernet and USB connectors and quad power outlets are all filtered so that RF gets

    neither in nor out.

    That's a SNB-installed bamboo rod running across the top of the chamber. It is used to hang the power and Ethernet

    DUT cables so that they don't get hung up as the turntable rotates the DUT. It's also handy to hang an LED worklight

    used when setting up routers for test. octoScope provided a plastic crossbar for this, but it was too tall and barely

    missed the top of the ASUS RT-AC68U's antennas.

    NETGEAR R8000 router under test inside the upper test chamber

    I ran many experiments before settling on the antenna configuration. I found the larger octoBox tended to yield

    lower throughput than the smaller box, especially for the 5 GHz band. Experiments showed the lower throughput was

    primarily due to higher path loss from the greater distance between the DUT and chamber antennas.

    I found the slightly closer antenna spacing helped compensate for some of the distance-caused path loss. The

    antenna array is centered on the chamber turntable. Distance from center of turntable to chamber antennas is 18

    inches (45.72 cm).

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/octobox_38_upper_chamber.jpghttp://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    6/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 5 www.octoscope.com

    Tel.: +1.978.222.3114

    Upper chamber antenna detail

    Speaking of which, theoctoBox-TTis a new feature in the upper chamber, used to rotate the DUT during testing. Its

    exact use is described inHow We Test Wireless Products - Revison 8.The center of the turntable is approximately

    XX" from the chamber antennas.

    Between

    The three antennas are connected to octoScope's new quadAtten programmable attenuator used to adjust signal

    level to simulate distance. The quadAtten can be seen in the first photo above, mounted on the side of the upper test

    chamber. The quadAtten outputs are connected to a SMA connectors on theoctoBox MPE,which sits between the

    upper and lower chambers. The MPE emulates multipath using IEEE 802.11n/ac indoor channel models B or C. The

    lower set of SMA connectors on the MPE connect directly to the lower test chamber, which holds the

    standard wireless test client (STA).

    Lower Chamber

    The V2 testbed retires the ASUS PCE-AC66 AC1750 class PCI-e board used as the reference client in V1 testbed.

    After careful consideration, NETGEAR's R7000 Nighthawk running firmware V1.0.3.60_1.1.27 configured in client

    bridge mode was selected as the new testbed reference client.

    The R7000 is directly cabledto the MPE using dual-shielded cables. Direct cabling is necessary to ensure high

    enough signal levels in both 2.4 and 5 GHz so that maximum link rates can be reached. Since the R7000 is an

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.octoscope.com/English/Collaterals/Documents/octoBox_Turntable_Datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_Turntable_Datasheet.pdfhttp://www.octoscope.com/English/Collaterals/Documents/octoBox_Turntable_Datasheet.pdfhttp://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.octoscope.com/English/Products/octoBox_MPE/octoBox_MPE.htmlhttp://www.octoscope.com/English/Products/octoBox_MPE/octoBox_MPE.htmlhttp://www.octoscope.com/English/Products/octoBox_MPE/octoBox_MPE.htmlhttp://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/octobox_38_antenna_detail.jpghttp://www.octoscope.com/English/Products/octoBox_MPE/octoBox_MPE.htmlhttp://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.octoscope.com/English/Collaterals/Documents/octoBox_Turntable_Datasheet.pdfhttp://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    7/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 6 www.octoscope.com

    Tel.: +1.978.222.3114

    AC1900 class router, we now can test the 2.4 GHz side of AC1900 routers to their full potential. The R7000 is

    connected via Gigabit Ethernet to the Dell Optiplex. The R7000 is managed via web browser from the lower

    chamber computer. The computer is accessed remotely via VNC.

    Lower Test Chamber view

    That concludes the tour of the V2 testbed.How We Test Wireless Products - Revison 8describes how we use it.

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/octobox_38_lower_chamber.jpghttp://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    8/14

    Reprinted by octoScope, Inc. www.octoscope.com

    +1.978.222.3114

    How We Test Wireless Products - Revision 8TUE, 29 JUL 2014 06:00

    TIM HIGGINS

    Introduction

    NOTE:This article describes the wireless test procedure for products tested after July 25, 2014.

    For wireless products tested between April 15, 2013 and July 24, 2014,see this article.

    We described the technology used in our new wireless test process inthis article.Now it's time to get into the exact

    details of how we use the new testbed.

    Environment

    Our V8 process is similar to the V7 process in that it uses octoScope anechoic chambers to provide a repeatable RF

    controlled test environment. The V2 testbed used by the V8 process has a turntable, which plays a key role in the

    new process.

    New SmallNetBuilder Wireless Testbed

    http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.smallnetbuilder.com/wireless/wireless-howto/32082-how-we-test-wireless-products-revison-7http://www.smallnetbuilder.com/wireless/wireless-howto/32082-how-we-test-wireless-products-revison-7http://www.smallnetbuilder.com/wireless/wireless-howto/32082-how-we-test-wireless-products-revison-7http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/wireless_testbed_v2_block.jpghttp://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2http://www.smallnetbuilder.com/wireless/wireless-howto/32082-how-we-test-wireless-products-revison-7http://www.smallnetbuilder.com/wireless/wireless-features/32478-how-we-test-wireless-products-revison-8http://www.octoscope.com/
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    9/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 2 www.octoscope.com

    Tel.: +1.978.222.3114

    The V8 process has been designed to address some weaknesses of the V7 process. Specifically:

    Small separation of DUT and chamber antennas

    Bias toward products with vertically polarized antennas

    The need to eyeball a "best" out of four test runs

    Inability to fully test AC1900 class products in 2.4 GHz

    The V8 process uses the same attenuation settings for "Location" values that are used in some benchmarks for

    backward-compatibility with products tested with old open-air methods.

    Key values for 2.4 GHzare shown in the plot below. Throughput at 0 dB attenuation is used for Maximum Wireless

    Throughput Ranking and throughput at 60 dB attenuation is used for Wireless Range Ranking in

    the Routerand Wirelessproduct Rankers.

    octoBox MPE Test Location Attenuations - 2.4 GHz

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/2-4ghz_mpe_test_points.jpghttp://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    10/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 3 www.octoscope.com

    Tel.: +1.978.222.3114

    Key 5 GHzvalues are shown in the plot below. Throughput at 0 dB attenuation is again used for Maximum Wireless

    Throughput Ranking and throughput at 39 dB attenuation is used for Wireless Range Ranking in

    the Routerand Wirelessproduct Rankers. Less attenuation is used for range ranking in 5 GHz vs. 2.4 GHz due to

    higher path loss at those frequencies.

    octoBox MPE Test Location Attenuations - 5 GHz

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/5ghz_mpe_test_points.jpghttp://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    11/14

  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    12/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 5 www.octoscope.com

    Tel.: +1.978.222.3114

    And here is an ASUS RT-AC68Uin its starting test position. (The white object is an LED work light removed during

    testing.) Power and Ethernet cables are routed to the DUT from above, to not interfere with DUT rotation.

    ASUS RT-AC68U in Upper Test Chamber - "0 degree" Position

    The router or AP under test is set as follows:

    Reset to factory defaults

    WPA2/AES security enabled

    2.4 GHz band: 20 MHz bandwidth mode, Channel 6

    5 GHz band:40 MHz bandwidth mode for N devices, 80 MHz bandwidth mode for 802.11ac devices,Channel 153

    We continue to use 20 MHz bandwidth mode for 2.4 GHz testing. Most environments have too many interfering

    networks for reliable 40 MHz operation.

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/octobox_38_asus_rtac68u_on_turntable.jpghttp://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    13/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 6 www.octoscope.com

    Tel.: +1.978.222.3114

    Test Process

    The general test process is as follows:

    1. Check / update latest firmware

    2. Reset router to defaults

    3. Change router settings:

    o LAN IP: 192.168.1.1

    o 2.4 GHz: Ch 6, Mode: 20MHz, WPA key: 11111111

    o 5 GHz: Ch 153, Mode: Auto or 80 MHz, WPA key: 11111111

    4. Associate client. Check connection link rate.

    5. Run IxChariot manual test to check throughput

    6. Run test script

    7. Power cycle DUT and repeat steps 4-6

    Two test runs are made in each band for dual-band devices.

    Testing is controlled by a Tcl script modified from a standard script supplied by octoScope. The script executes the

    following test plan:

    1. Move the turntable to starting position. This places the DUT at the "0 degree" starting position previously

    described.

    2. Program the attenuators to 0 dB

    3. Move the turntable to -180 (counter-clockwise)

    4. Start 90 second IxChariot test (throughput.scrwith 5,000,000 Byte test file size) simultaneous up and

    downlink (0 dB only)

    5. Wait 30 seconds, then start turntable rotation to +180 at 1 RPM.

    6. Wait for test to finish

    7. Discard first 30 seconds of IxChariot data, calculate average of remaining data and save to CSV file. Save

    entire IxChariot .tst file.

    8. Repeat Steps 3 to 7 for downlink only (DUT to STA)

    9. Repeat Steps 3 to 7 for uplink only (STA to DUT)

    10. Increase attenuators by 3dB.

    11. Repeat Steps 3 to 9 until attenuation reaches 45 dB for 5 GHz test; 63 dB for 2.4 GHz test.

    We continue to use thethroughput.scrIxChariot script instead of the High_Performance_Throughput.scrscript used

    by many product vendors.

    The main difference is the high performance script uses a 10,000,000 Byte default test file size, Microsoft's

    Overlapped IO socket and sets the transmit and receive buffer sizes to 64 K bytes. The regular throughput script

    doesn't use Overlapped IO, leaves the transmit and receive buffers at default and a 100,000 Byte test file size.

    We found in testing that using the throughput script and adjusting the file size upward actually produced slightly

    higher throughput than the high performance script.

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-
  • 8/10/2019 OctoScope SmallNetBuilder's Wireless Testbed

    14/14

    How We Test: SmallNetBuilder's Wireless Testbed V2

    Reprinted by octoScope, Inc. 7 www.octoscope.com

    Tel.: +1.978.222.3114

    The CSV files from the two test runs are merged into a single Excel file and the average of the two runs calculated.

    The averageof the two runs is the value entered into the Charts database.

    Test Notes

    Moving the DUT through an entire 360 while the IxChariot test is running removes any orientation bias from the test

    and eliminates the need to determine the "best" run out of multiple fixed-position tests. Because throughput typicallyvaries during the test run, the averaged result tends to be lower. Rotation at lower signal levels, particularly in 5 GHz,

    frequently causes the connection to be dropped sooner than if the DUT were stationary. For this reason, V8 5 GHz

    test results will generally not extend beyond 39 dB.

    The IxChariot plot below shows a typical test run. The first 30 seconds of the run include a throughput step-up that is

    caused by an IxChariot artifact and does not reflect true device performance. The first 30 seconds also provides time

    for DUT rate adaptation algorithms to settle. For both reasons, the test script excludes the first 30 seconds of data in

    the average value saved in the Router Charts database.

    IxChariot Test Data detail

    Our testing focus has always been to provide results that provide the fairest relative comparisonamong products.

    This new process achieves that goal by removing multiple sources of test setup bias to achieve even fairer product-

    to-product comparison.

    http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-http://www.octoscope.com/http://www.octoscope.com/http://www.smallnetbuilder.com/images/stories/wireless/how_to_test_wireless/wireless_test_360deg_rotation.jpghttp://www.octoscope.com/http://www.smallnetbuilder.com/wireless/wireless-features/32477-how-we-test-smallnetbuilders-wireless-testbed-v2-