nts video streaming compatibility / functionality test outline

NTS Testing Labs Video Streaming Test Lab

Compatibility & Functionality Test Outline

Revision 1.0

NATIONAL TECHNICAL SYSTEMS

The NTS Mission:

Assisting our Clients in Navigating a Short Course to Global Markets

Written by: Ferdinand Hidalgo, Project Manager

Tracking No. TO201007

Rev. 1.0

Approved by: Osman Sakr

Date 2/22/02

Quality Assurance Approval Derek Fakehany

Date 2/22/02

FileName NTS Test Outline - Video Streaming - TO201007.doc

Copyright © 2002, National Technical Systems, Inc. This document is provided to clients and potential clients of National Technical Systems (NTS) for informational purposes only. It may not be reproduced or stored, in whole or

in part by any means, with out the express written permission of NTS.

Document Tracking # TO201007 NTS Test Outline - Video Streaming - TO201007.doc 1.0 02/02

Table of Contents 1. Introduction .........................................................................................................................................................1 2. Scope of Work .....................................................................................................................................................1 2.1 Overview ...........................................................................................................................................................1 2.2 Test Procedures................................................................................................................................................1

2.2.1 Hardware Test Methodology.........................................................................................................................1 2.2.2 Browser Test Methodology ...........................................................................................................................2 2.2.3 Testing Under Bandwidth Limitations............................................................................................................2 2.2.4 Server Test Methodology..............................................................................................................................3

3. Test Environment ................................................................................................................................................3 4. Project Deliverables ............................................................................................................................................4 4.1 NTS Test Reports .............................................................................................................................................4 4.2 Test Matrix ........................................................................................................................................................4 4.3 Project Documentation Report ..........................................................................................................................4 4.4 Report Transmission.........................................................................................................................................5 4.5 Project Leadership ............................................................................................................................................5 5. Conclusion...........................................................................................................................................................5

i

Error! Reference source not found. Video Streaming Test Outline

1. Introduction National Technical Systems (NTS) Testing Laboratories currently conducts Video Streaming Compatibility / Functionality Testing for a wide array of product manufacturers. Video streaming testing will be conducted in four phases: Hardware Test Methodology, Browser Test Methodology, Testing Under Bandwidth Limitations, and Server Test Methodology. Video Streaming Compatibility / Functionality Testing fills the need for this “real-world” environment testing. The following Test Outline will describe the methodologies and test procedures used in Video Streaming Compatibility / Functionality Testing.

2. Scope of Work

2.1 Overview NTS typically conducts Video Streaming Compatibility / Functionality Testing to verify that the product under test functions correctly without anomalies, difficulties or discrepancies. NTS will perform testing to exercise a video streaming product to insure that it will function correctly. A streaming video test will cover the following objectives:

◊ ◊ ◊ ◊ ◊

Hardware compliance with different third-party peripherals Browser compliance with a mix of different third-party applications Ensure the operating system behaves according to the client’s specifications Testing under bandwidth limitations Server compliance with different types of peripheral and stream media applications

If critical errors are encountered, then the product will fail its quality assurance test. If no critical errors were encountered, then the product will pass its quality assurance test.

2.2 Test Procedures Before NTS can begin testing, equipment must be setup and configured with the appropriate operating system, peripherals, etc. Once the test configuration is configured, NTS Testing Engineers will confirm that the test configuration is functioning correctly without anomalies. The quality assurance test consists of four (4) test areas: Hardware Test Methodology, Browser Test Methodology, Testing Under Bandwidth Limitations, and Server Test Methodology. Below are those quality assurance areas and the test procedure outlines that coincide with each area.

2.2.1 Hardware Test Methodology Different test platforms will be setup to perform functionality and compatibility tests with the following test platforms and hardware components. Server Specifications Depending upon the type of streaming needed, the following platforms are an example of what may be used during video stream testing:

File Server (Stored Video Files) NTS will use a file server configured with Microsoft Windows NT v4.0 or Microsoft Windows 2000 Server as an external storage device. The movies will be saved on a mapped directory – the files will be downloaded using the test platform but they will be stored on the file server. The NTS Test Engineers will verify the time and quality of the downloaded movies. The NTS Test Engineers will replay the movie from the file server later on.

February 22, 2002 National Technical Systems TO201007-1.0 - 1 -


Streaming files servers can also be used when installing these operating systems depending on which streaming software will be used:

◊ ◊ ◊

Microsoft Windows NT v4.0 Server Microsoft Windows 2000 Server Apple MacOS X

Streaming Media Server Streaming-media server software can be hosted locally or outsourced. Servers will be configured based on client specifications, which can include operating system and bandwidth settings

Graphics / Video Cards NTS will use different video cards to verify the performance of each one of the movies being played on the Internet or after downloading the movies. The test will focus on how good the movies can be displayed under different video cards and when executing them simultaneously with many applications open at the same time. Sound / Audio Cards Different sound cards will be used to verify how the audio of each movie can be played. Audio will be verified to test clarity, smoothness, and completeness of audio.

2.2.2 Browser Test Methodology Browser compatibility test focuses on detecting any anomalies on the website. Different versions of Internet Explorer and Netscape browsers will be used for this test under different Operating Systems. For Streaming Video the following features on a website will be covered:

◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊

◊ ◊ ◊ ◊ ◊

◊ ◊ ◊ ◊ ◊ ◊ ◊

Did the user encounter any problems with the download interface of the movies? Does the user needs plug in such as Flash, Shockwave to view to play the movie online? Verify which browser is more compatible with the website. Verify the performance of the video of each movie under different video resolutions. Verify the quality of sounds – verify the quality of the sound online and after downloading the files. Verify if the movies can be played using Microsoft Media Player, QuickTime and Real Networks. Verify how easy are the instructions on how to download or play movies from the website. Verify how navigation of the website – how easy, comfortable the user can download the movies.

2.2.3 Testing Under Bandwidth Limitations As part of the streaming video testing, our test engineers will stream movies using a dial up connection and/or T1 connection. This test will focus on the quality of the video stream with higher bandwidth and low-bandwidth limitations under:

Modem 56K modem ISDN Cable Modem DSL T1

Overall Quality of Streaming Video

Sharpness of Video, Pixelation Choppiness, dropped frames Discoloration Evenness of screen Clearness of audio Smoothness of audio Changes in volume


Error! Reference source not found. Video Streaming Test Outline 2.2.4 Server Test Methodology This test focuses on setting up different servers and verifying how reliable they are under multiple download sessions. NTS will exercise the product under testing with streaming video using Windows Media Player, QuickTime and Real Networks:

1. Microsoft Windows Media Technologies v8.0 2. Apple QuickTime v5.0 3. Real Networks Products

Microsoft Windows Media Test

◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊

◊ ◊ ◊ ◊ ◊ ◊

◊ ◊ ◊

◊ ◊ ◊ ◊

Setup a Microsoft Windows 2000 server with IIS5.0 with Windows Media Services 8 Setup a Windows NT 4.0 Server with Windows Media Services 8 Test the Platform with Windows Media Encoder 8 Increase and decrease the volume of the speaker – verify the quality of the sound of the movies Verify the quality of the audio under different sound cards Set the different video rates on the server in which each video can be downloaded Set the different video rates on the server in which each video can be downloaded Adjust the maximum bit rate of file in which the file can be downloaded Verify the performance of the server using a 56.6k connection, ISDN, Cable, DSL and T1 connection

Real Networks Media Test

Setup a Microsoft Windows 2000 or Microsoft Windows NT v4.0 Install Real System Server Plus 8 or RealServer Basic 7 Verify if each server is able to handle many downloads of video at the same time Verify if the movies can be downloaded via http and Surestream Adjust the encode bandwidth range, from 28.8 Kbps to LAN Verify if at lower bandwidths, the RealNetworks content was smoother and less blurry.

Apple QuickTime Test

Setup a Apple PowerMac with at least 128MB of RAM and Apple MacOS 9.0 Install the QuickTime Streaming Server 2.0.1 Install some platforms with Microsoft Windows 95, Microsoft Windows 98, Microsoft Windows Millennium Edition, Microsoft Windows 2000, Apple MacOS 8, Apple MacOS 9 and Apple MacOS X Verify if each movie can be displayed using QuickTime Test the server using different types of connections such as dial-up 56.6k, ISDN, Cable, DSL, and T1 Verify the quality of the sound Verify how smooth the video is played

3. Test Environment Below is an example listing of test platforms and operating systems typically deployed by NTS when performing compatibility testing. When configuring the test environment, NTS Testing Laboratories selects platforms and operating system configurations which represent “real world” conditions. Sample of PC Test Platforms

Test Platform Operating System 1. Compaq Presario 2286 Cyrix MII-333 MMX Microsoft Windows 98 SP1 2. Acer AcerPower 8600 Pentium III/800 Microsoft Windows Millennium Edition 3. Acer Aspire 3230 AMD K62/366 MicroTower Microsoft Windows Millennium Edition 4. Compaq Presario 7998 AMD Athlon/1GHz Microsoft Windows 2000 Professional 5. IBM Aptiva K7 650Mhz Microsoft Windows 98 Second Edition 6. Acer TravelMate 737TLV P3700 - Microsoft Windows 2000 Professional 7. Compaq: Presario 1275 AMD K6-2 366MHz Microsoft Windows 2000 Professional



Test Platform Operating System 8. Dell Latitude 3500 CPi300XT Pentium II/300Mhz Microsoft Windows 98 Second Edition 9. Hewlett-Packard Pavilion 8650 Celeron/533 Microsoft Windows 2000 Professional 10. Hewlett-Packard Vectra VL8 Pentium II/450 Microsoft Windows 98 Second Edition 11. AST Bravo 500A-1 Pentium III/500MHz Microsoft Windows 98 Gold Edition 12. Compaq Presario 5900Z AMD Athlon/700Mhz Microsoft Windows 2000 Professional 13. Compaq Prosignia 310 Celeron 400Mhz Microsoft Windows 98 Gold Edition 14. Gateway G6-400C Intel Celeron/400 Microsoft Windows 98 SP1 15. Compaq ProSignia 330 Pentium III/500Mhz Microsoft Windows 98 16. Hewlett-Packard Omnibook 7150 PII/300 Microsoft Windows Millennium Edition 17. Hewlett-Packard Brio MT Celeron/333MHz Microsoft Windows 98 Gold Edition 18. Hewlett-Packard Pavillion 4430 AMD K6-2/300 Microsoft Windows 98 Second Edition 19. Hewlett-Packard Pavillion 6460 Celeron/400 Microsoft Windows 98 SP1 20. Hewlett-Packard Pavillion 9695C AMD Athlon/850 Microsoft Windows 2000 Professional

Sample of Macintosh Test Platforms

Test Platform Operating System 1. Apple PowerBook G3 PPC G3/400 Apple MacOS X 2. Apple iBook Graphite 466MHz Apple MacOS 9.1 3. Apple iMac 333 MHz Apple MacOS X 4. Apple iMac G3 Power PC 233 MHz Apple MacOS 8.6 5. Apple PowerMac G4 Cube/450 Apple MacOS 9.1

4. Project Deliverables NTS clients receive the following standard NTS project deliverables during or at the conclusion of tests performed by NTS. (Project specific deliverables may also apply depending upon the specific project type).

1. NTS Test Reports (daily / as generated) 2. Test Matrix and/or Test Status Report 3. Project Documentation Report

The Test Matrix and NTS Test / Problem Reports are transmitted to client daily or as developed. The Project Documentation Report is delivered following the conclusion of testing.

4.1 NTS Test Reports During testing, NTS generates reporting to document all testing performed. NTS test reporting is designed to capture configuration information, general procedures, test measurements and other test results related to all tests performed.

4.2 Test Matrix The NTS Test Matrix provides a quick overview of the status of entire project. The Test Matrix is a master grid outlining individual tests or test milestone coverage on the project. The matrix provides a quick overview of tests performed to date, with columns specifying the product(s) tested, a pass/fail/warn status, the date performed, test engineer identification, and any related Test Report numbers.

4.3 Project Documentation Report NTS binds the final copy of the project test matrix along with all NTS test reports and delivers this to clients following the completion of testing.


Error! Reference source not found. Video Streaming Test Outline 4.4 Report Transmission NTS uses Internet e-mail as much as possible to transmit status reports, test reports, test matrices, documentation reports, etc. In some cases, materials may also be faxed or send via FedEx. If additional security is required, NTS will utilize encryption to prepare and send files in batches, using an agreed upon encryption mechanism (to be decided between NTS and a particular client prior to test execution). Catastrophic and critical problems are prioritized and reported upon as soon as possible.

4.5 Project Leadership A project leader is assigned to projects NTS performs for clients. The NTS project leader communicates with clients regarding test progress, status, and findings. The project leader’s primary responsibility is to insure that testing is being executed as agreed upon, to provide management and technical expertise, and to work with the Client to make any required changes. The project leaders are supervised by NTS’ Operations Manager. If major revisions to a test plan is required, or if a client is not satisfied with the testing process, the client is encouraged to contact the Lab Quality Manager immediately. It is NTS’ goal to provide testing that meets Client’s needs as accurately as possible.

5. Conclusion NTS Testing Laboratories has decades of experience and unsurpassed expertise in the area of compatibility and interoperability testing. Fewer product recalls, faster time to market, reduced technical support requirements and higher customer satisfaction are but a few of the many potential benefits of testing a product with NTS. Video streaming compatibility and functionality testing fill the need for “real-world” environment testing. Testing the video streaming product under the functionality areas of Hardware Test Methodology, Browser Test Methodology, Testing Under Bandwidth Limitations, and Server Test Methodology will assure that the client’s product will function in a “real world” environment. The benefactors of this type of compatibility and functionality testing are lower product recall rates, lower volume of customer support calls and higher consumer satisfaction.


NATIONAL TECHNICAL SYSTEMS

National Technical Systems (NTS) is one of the largest consulting and certification laboratories in the United States. NTS operates extensive test facilities, throughout the United States and Japan, covering operational environment, stress environment, compatibility, interoperability, broadband network testing, EMI/EMC, FCC, NEBS, USB, WHQL, Product Safety testing and ISO Audit/Certification (we certify the National Space Administration for ISO compliance). NTS is a Nationally Recognized Testing Laboratory (NRTL) - as certified by the United States Government. We are accredited to ISO 17025 by the American Association for Laboratory Accreditation. NTS holds many certifications from foreign governments as a Conformity Assessment Body (CAB). We are the only “one stop” certification and compliance facility offering a broad range of testing capabilities. These capabilities include significant financial resources, numerous certification programs and a high level of lab security. We support extensive test procedures, operating systems and hardware. We are fully equipped for broadband testing and offer such services as DOCSIS “third-party” pre-certification and DSLAM compatibility and functionality testing. A variety of PSTN and Internet access methods are available ranging in speed from dial-up to 1Gb/sec. Several independent T1 Internet access points and frame relay circuits are available. An OC-12 link permits access ranging from standard T1 to OC-12 (644 Mb/sec) in a variety of formats: ATM, Frame Relay, SuperTrunk, HiCap and various other architectures. Our particular testing laboratory is located in Culver City, California. This facility is centrally located and is just 15 minutes from Los Angeles International Airport. IT IS EASY TO START! Send us your “wish list” covering the areas that you would like addressed in your test, and we will design a test proposal specific to your needs that covers only what you want tested. To turn your request into a formal (no cost to you) proposal, we need the following:

1. Product description and or specification. 2. Target Market – What market your product will sell into. 3. Start and end date – When you need to start and end this test. 4. Company information – Name, contact, address, phone, fax and e-mail. 5. Any concerns about your product that you have or you know about.

Let us know if you want a test proposal for your particular project. Typically, our Proposal Department can prepare one for you within five (5) working days, although more complex products may require additional time. The proposal will contain our test methodology, project schedule, defect classifications, regression information, and the project cost. This proposal should be considered a “working document” that you can change or modify to fit your exact test requirement and budget – Our goal is to work together so that you can achieve your objectives. CONTACT US National Technical Systems Computer Testing Division 5730 Buckingham Parkway Culver City, CA 90230 Telephone: (310) 641-7700 Fax: (310) 641-5827 Web: http://www.ntscorp.com/ E-mail: [email protected]

http://www.ntscorp.com/

mailto:[email protected]

nts video streaming compatibility / functionality test outline

Documents