introduction to wireless product acceptance optimization solution-v0.3-20091111.ppt

7/29/2019 Introduction to Wireless Product Acceptance Optimization Solution-V0.3-20091111.ppt

http://slidepdf.com/reader/full/introduction-to-wireless-product-acceptance-optimization-solution-v03-20091111ppt 1/41

HUAWEI TECHNOLOGIES CO., LTD.

www.huawei.com

Huawei Confidential

Security Level:2013/9/12

Introduction to Wireless

Product Acceptance

Optimization Solution



Contents

• Overview

• Solution Introduction

•

FAQs• Negotiation Cases



HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential

Difficulties in Project Acceptance (Test)

The Ac ceptance Manual is a collection of test cases. For specific projects, however,

many cases may be redundant. On-site

engineers may not know how to identify and

delete redundant cases or choose test cases.

Therefore, risks of uncertainty exist.

Acceptance supporting documents (such as

factory inspection reports and R&D internal test

reports or documents) are missing or hard toobtain.

Existing test case selection tools cannot select

suitable cases correctly like on-site

engineers.

Acceptance contract clauses are unclear.

Disputes with customers in later delivery phase

puts us in an unfavorable position.

Acceptance standards are determined throughnegotiations with customers. However, we lack

negotiation strategies or proper measures

when customers raise unreasonable

requirements.

1 Materials: Hard to select proper test

cases among massive test cases

2 Documents: Lack supporting

documents or hard to obtain

3 Tools: Inapplicable to guide on-site

engineers to select proper test items

Five shortcomings in ability development

4 Clauses: Unclear clauses causing

disputes with customers

5 Strategies: No guidance strategy for

negotiations




How Does This Solution Help

Focusing on product acceptance (provisional acceptance), this solution solves

problems related to materials, supporting documents, tools, clauses, and

negotiation guidance strategies, minimizes the on-site acceptance, shortens the

acceptance period, and fastens payment collection.

Materials:

Optimize acceptance information

and further classify test packages

in acceptance scenarios

Strategies:

Define acceptance negotiation

guidance strategies to support

on-site negotiations

Documents:

Prepare supporting documents and

ensure that the documents can beobtained by on-site engineers

Clauses:

Draft acceptance clauses

to restrict customer test

requirements

Tools:

Develop auxiliary tools, associate

feature sales and acceptance, and

enhance test case risk management

Minimize the acceptance, shorten the acceptance period, and fasten payment

collection



Contents

• Overview


•

FAQs• Negotiation Cases




Solution Introduction

Test Packages

Acceptance Supporting Documents

Test Case Selection Tools

Clauses Negotiation Guidance Strategies




Classifying the Test Cases in the Acceptance Manual

into Test Packages

Divide the original

Acceptance Manual into

four test packages and one

strategy package according

to test case attributes and

test procedures.




Structure of a Test Package




Associating Test Packages with Acceptance Process

Accepta

nce

Activity

• Factory

inspection report

(optional)

• Industry

certification

report (optional)Packing list

• List of materials

……

• Product

function/feature test

• Network

deployment service

acceptance

• Single-site

verification

•

Equipmentinstallation report

• Self-check report

……

• Network KPI

test

• System KPI

test ……

• Outstanding

problem

settlement

• Trial run

Test

package

<Specialized

certification

package>

<Feature verification

package>

<Site acceptance

package>

<KPI test

package> －

<Strategy package>

SupplyAcceptance

Provisional Acceptance Network

Optimization

Acceptance

FinalAccepta

nceDemonstration

AcceptanceSites Acceptance

Acceptance

Process

The provisional acceptance is classified into demonstration acceptance and sites acceptance.

The demonstration acceptance is used to test functions and features of products. The sites acceptance is used to test

engineering service quality and basic services.

Demonstration acceptance and sites acceptance can be performed in any order or at the same time.

Note: Engineers are not prompted to perform demonstration acceptance unless customers require

testing product functions at the acceptance stage.




Associating Test Packages with Acceptance Process-

Demonstration Acceptance Demonstration acceptance is a new concept aiming to complete product

function/feature tests within a small range and avoid large-scale tests siteby site. The small range can be test beds, trial sites, and validation

networks.

Test Bed

NodeBRNC

BTSBSC

Laboratory

RNCNodeB

Overall Network

Trial Sites Within an Overall Network

Trial Sites

Type A

Type BType C

RNCNodeB

RNC

NodeB

Overall Network Validation Network

Validation Network

• The test bed is a laboratory simulated

network. The hardware and software

configurations of this network are the same

as those on the actual network and can

represent the functions of the actual

network.

•Trial sites refer to

certain sites to be

constructed on the

network. The

hardware andsoftware

configurations of these

sites are the same as

those on the actual

network and can

represent the

functions of the actual

network.•The validationnetwork refers to a

subnet of the networkto be constructed. Thehardware andsoftwareconfigurations of thissubnet are the sameas those on the actualnetwork and canrepresent thefunctions of the actualnetwork.




Test Package Delivery Owner and Precautions

Test PackageDelivery

Owner Precaution

Specialized

certification package

Supply

chain/R&D

1. Optional. If customers have test requirements, provide the factory inspection

report first.

2. If customers require tests, guide customers to allow tests at Huawei

headquarters. In this case, quotation is required.

Feature verification

package

GTS/R&D

1. Optional. If customers have the requirement, provide internal test reports in

priority to avoid repeated tests.2. If tests must be performed on site, control the tests within a small range, for

example, in test beds, trial sites, and validation networks, to avoid large-scale tests

site by site.

3. Some test cases may fail to be verified due to immature industry chain,incomplete product implementation, unobservable effect, or theoretical ultimate

capacity. For such test cases, only a commitment to provide clarification

documents (emulation and theory analysis) can be made.

Site acceptance

packageGTS

1. Guide customers to perform tests based on a certain percentage, for example,

10%.

2. If tests involve expensive devices, quotation is required or customers provide the

devices.

KPI test package GTS This solution focuses on device acceptance (provisional acceptance) withoutinvolving network acceptance. Only KPI test cases are provided.

Strategy package －

Customers may have test requirements beyond the acceptance range, for

example, IOTs, RF tests, new solution/feature tests, and network acceptance. A

strategy package contains relevant guidance strategies to avoid impacts of the

preceding tests on the acceptance.





Test Packages








•The acceptance supporting documents include two test packages and four types of reports.




End-to-End Feature Test Report

End-to-end process of a feature

Feature

• R&D engineers shall

provide test reports on theoptional and special (test

cases of features hard to

be executed) features in

the feature verification

package in priority. The test

reports of all features will

be finally available when

the features are released.

• These reports can be

shown to customers during

negotiations, reflecting the

strict product quality test

procedures of Huawei and

persuading customers toavoid repeated tests during

product acceptance.

Step 1: R&D

engineers design test

cases based on

features and provide

internal test reports or

documents.

Step 3: On-site engineers

negotiate with customers to

avoid tests based on the

internal test reports and

documents provided by R&D

engineers.

Step 2:

Marketingpersonnel

sell features.

ReleaseSales

Service

delivery




How to Obtain Acceptance Supporting Documents

Factory inspection report

Wireless product contact person: Zeng Jun (ID: 00106221), Wireless Test & ATE Product Dept

Download address: http://10.72.128.225/texpert/Default.aspx Industry certification report

See the External Document Archiving Database Query Guide V2.0.ppt .

Download address:

http://support.huawei.com/support/pages/kbcenter/view/product.do?actionFlag=detailProductSimple&we

b_doc_id=SC0000526336&doc_type=123-2

Version test report and feature test report

These reports are released together with test cases (test packages). Download address:1. Chinese edition (including strategy guidance):


b_doc_id=SC0000526336&doc_type=123-2

2. English edition:


b_doc_id=SE0000414297&doc_type=123-2

Note: Industry certificates are applicable to all kinds of products. Different release test reports

and feature test reports are available for different kinds of products. Only the download address

of the UMTS test package is given. See Appendix for download addresses of test reports of

other products.





Test Packages







Key Improvements to Test Case Selection Tools

1. Risk management of feature

test cases

Feature test cases are packaged

according to the difficulty of test

execution, and test risk degrees

are marked by "basic", "optional",

and "special".

2. Associate front end (sales)and back end (acceptance

and delivery) and provide

automatic links to test cases

from Feature ID or Model in

BOQ.




Case Management in Excel Format—Easy to View Test

Case Details

View the details of a test case by

clicking Test Name

• Able to know details of a test caseat any time

• Easy to execute tests and select

test cases




Case Management in Excel Format—Select Test Cases to

Generate Word Files

Use macro to generate a test case and a signature form in Word format

1. A test cases is selected except that the Select flag is null or set to No or Cancel.

2. The editing operations are similar to those of an ordinary Excel sheet, for example,

copy, paste, add, and delete. Run macro to generate a test case description file

and a signature form in Word format.

S i gn a t ur e

F or m

A

c c e p t an c e

M

an u al




Case Management in Excel Format—Execution Result

Identifier and Statistics

Automatically generate test case

execution results

• Test case execution results are

marked in different colors.• Execution result statistics are

performed automatically on a

daily or weekly basis.




Case Management in Excel Format—Test Case Selection Tools

Select test cases automatically:

1. In the Selecting Test Case dialog box, determine the baseline document (Excel format) and test case package asprompted.

2. Set conditions. As shown in the figure above, select the test cases whose Model in BOQ is included in the model list and

Priority is High. After the test cases meeting preset conditions are selected, marks are automatically filled in Selection

Mark of the selected test cases. You can start a new search and combine the search results or start another search in the

search results.

3. The selected test cases can be used to generate test case description files and signature forms in Word format.





Test Packages







Technical Clauses Regarding Acceptance

No on-site acceptance is required for items that are clearly specified in standards

or protocols. (For example, for sensibility and standard signaling, third-partycertification reports and compatibility analysis reports can be provided to replace

tests.)

Tests that require special test instruments (such as power meters, special

spectrum testers, and voice/image/stream quality evaluation tools) are not

performed on site. No acceptance is required for ultimate functions (for example,

xx subscribers in a single cell). (Internal test reports can be provided for

clarification, or witness tests can be performed with R&D engineers for important

items).

No commitments shall be made to verify all sold features on site. (Some customers

even request us to test the features of other vendors. Internal test results and

clarification documents can be provided to replace the feature verification.)

No commitments shall be made to test items that require third-party cooperation

(for example, terminal interworking and service platform compatibility; MVI test

services can be provided to meet carrier requirements). Standard test cases can

be provided. Test services can be sold separately to meet extra test requirements

of customers.




Acceptance Responsibility Division Clauses

Clearly specify that the carrier is the responsible person for

equipment interconnection and the equipment supplier is

responsible for solving equipment problems. (In principle, no IOT is

performed during acceptance.)

Clearly specify that the carrier should prepare all third-party toolsand devices required for tests (mainly trace tools). (In principle, no

other test tools are recommended. Guide customers to prepare the

tools.)

The acceptance certificate is still acquirable if the acceptance isblocked or delayed by non-supplier reasons.




Notes:

Application scenarios of clauses: At the bidding stage, if acceptance standards are not clearly determined with customers,

clauses restricting the scope of tests shall be added in contracts to prevent some risks

and avoid being passive when confirming acceptance standards with customers later.

How to use the clauses: The technical clauses that are important and widely applicable should be included in main

contracts, usually in Condition part.

The technical clauses that are important and specific should be included as an

acceptance attachment of contracts, also in Condition part.

Responsibility division clauses are included as an acceptance attachment in Roles and

Responsibilities part of contracts.；





Test Packages



Clauses

Negotiation Guidance Strategies




General Negotiation Principles

Use the fewest test packages to meet customer requirements.

1

Make clear that the acceptance aims to prove the normal service provisioning of equipment,

instead of verifying features.

2

Make clear the delivery modes of acceptance: certificates, test reports, clarification

documents, tests at Huawei HQ, and on-site tests.

3

Huawei shall provide paid test services which are claimed to be independent of acceptance

and undertaken as a separate project operation.

4

All test packages are concepts used internally and should be known as "acceptance

test packages" for customers.

5




Negotiation Scenario 1: All Feature Tests

Requirement Description Risk Analysis Recommended Guidance Strategy

Carriers require verifying

all features before the

equipment is put into

commercial use, which is

the most common

requirement.

Some carriers even

require verifying all

features at each site.

1. Features are

classified into very

small categories and

the on-site test

workload is too heavy.

2. Features of Huawei

are not clearly

classified, and some

features are hard or

even impossible to be

tested on site.

Strategy 1:

Using documents to

replace tests

1. Guide relevant engineers to provide release test

reports. The test reports should contain statistics data

that proves the verifications performed internally.

2. Guide relevant engineers to provide feature

verification reports, including test cases (logs should be

available in reports for tracking).

3. Test contents are within the range of baselines.

Clarification documents are provided if on-site

verification conditions are not met.

Strategy 2:

Performing tests within

a smaller range

On-site tests are allowed. The demonstrationacceptance mode is recommended, and the test

environment can be test beds, trial sites, or validation

network. No repeated tests at each site are permitted.

Strategy 3:

Performing acceptance

based on a certain

percentage (Note)

The acceptance at actual sites is allowed only in

sampling test manner, for example, based on a

percentage of 5% to 10%.

Note:

1. In principle, not all features need to be tested. Only the features that are purchased by customers

and to be put into commercial use need to be tested.2. Feature tests should not be performed at

actual sites. Strategy 3 is a compromise made with customers only when no other approach is

available.




Negotiation Scenario 2: IOTs

Requirement

DescriptionRisk Analysis Recommended Guidance Strategy

Generally, carriers require

that suppliers provide the

equipment supporting

interconnection with the

existing network equipment

and the previous

interconnection reports.

Some carriers even fail to

obtain support from original

suppliers.

The protocol compliance of

inter-equipment

interconnection cannot be

settled through only efforts

of Huawei.

With NDAs signed, IOT and

MVI reports shall not be

provided for a third party

without permission.

Due to competition

purposes, interconnection

equipment suppliers may

provide no cooperation or

even block the acceptance.

Strategy 1:

Using reports to

replace tests

If Huawei equipment is interacted with the

equipment of important carriers or mainstreamsuppliers, make NVIOT clarification with carriers

and guide carriers to raise the IOT report

requirement.

For interconnection requirements in other scenarios,

refer to MVI reports (without NDA) and commercial

application cases.

Strategy 2:

Restricting the testrange and specifying

responsibilities

Equipment interconnection must be completed for

network commissioning, but not all IOTs need to be

performed. Interconnection is required to meet onlycommercial application requirements through close

cooperation among three parties. The PAC is still

acquirable if the interconnection is delayed by non-

Huawei reasons.

Strategy 3:

Providing IOTs as a

test service

independent of

acceptance

If carriers require all IOTs, promote the selling of

MVI test services and obtain cooperation from

interconnected equipment suppliers.

Note:

The test service is a professional service that is operated independent of project acceptance and

used to meet various test requirements of customers. In terms of acceptance negotiation strategies,

the test service is a way to meet requirements and reach a balance with customers by means of

quotation.




Negotiation Scenario 3: Capacity Specification Tests

Requirement


Some carriers pay a great

concern to equipment

capacity and require

relevant tests.

Some carriers even requireverifying the ultimate

specifications in real

scenarios.

Capacity specifications are

usually the equipment

processing capabilities

obtained under theoretical

baseline conditions, which

are hard to be tested on site.

At present, the equipment

capacity is very large.

Ordinary test tools cannot be

used to obtain thespecifications of the main

equipment, special test tools

cannot be used for on-site

tests due to restrictions of

shipment, customs

clearance, costs, and

personnel skills.

Strategy 1:

Obtaining test-free

permission

Provide sufficient evidences showing the difficulty in

ultimate capacity specification tests and guide

customers to cancel such test requirements.

Strategy 2:

Using reports to

replace tests

Promise to provide internal test reports to reduce

on-site test workload.

Strategy 3:

Returning to Huawei

HQ for test

Guide carriers to perform witness tests with R&D

engineers at Huawei HQ.

Note:1. The changes of capacity specification tests are complex. No commitments shall be made to meet

special traffic model and networking requirements unless carriers perform the tests.

2. Any commitment related to tests involved in strategy 2 and strategy 3 shall not be made before

being confirmed and agreed by R&D personnel. Tests involved in strategy 3 shall be quoted.




Negotiation Scenario 4: RF Index Tests

RequirementDescription Risk Analysis Recommended Guidance Strategy

Some carriers require

performing RF index tests

on site, even including

electrical performance and

environment requirements.

Some tests are required to

comply with local standards.

Index tests require complex

test tools and high costs and

are hard to be performed on

site.

Test personnel are not

familiar with local standards,standards are inconsistent,

and certification costs are

high.

Strategy 1:

Providing international

organization

certification reports

Standard tests should comply with third-party

organization certification reports, and no on-site

tests are performed. Guide customers to provide

certificates or reports.

Replace local standards with corresponding

international standards.

Strategy 2:Providing factory

inspection reports

For test items that vary with modules, such asRF indexes, factory inspection reports can be

provided as the index satisfaction evidence.

Strategy 3:

Returning to Huawei

HQ for test

Guide carriers to perform witness tests with R&D

engineers at Huawei HQ.

Note:

1. For on-site tests, clearly specify that carriers are responsible for preparing test tools and

performing tests on only one or two modules.

2. Confirm strategy 2 with the production contact person; confirm strategy 3 with the R&D

contact person at Huawei HQ.




Negotiation Scenario 5: Deep Test Requirements

Requirement


During the acceptance

workshop, carriers perform

finitist and theoretical

analyses, increasing test

requirements abruptly.

Some carriers also have

networking, configuration,

and commercial strategies,

and even require FOA

tests.

The purpose of these tests

are to find commercial

application schemes, which

is already beyond the

acceptance scope.

The scope of FOA must be

strictly controlled;

otherwise, it may be mixed

with pre-commercial

application and the

commercial application may

be conducted before PAC is

obtained, making us

passive.

Strategy 1:

Performing

technical

clarification

Tests are not necessary so long as commercial

application schemes pass the technical clarification

during workshop.

Strategy 2:Conducting training

If test items are added for personnel training, the tests

can be differentiated from acceptance according totraining operations.

Strategy 3:

Providing "test

services"

If on-site tests are required, a balance can be reached

by selling "test services".




Overall Strategy Model

Strategy package(IOT and ultimate equipment capability tests)

Specialized certification

package(professional certification reports/tests)

Feature verification

package(wide product feature tests)

Site acceptance

package(service acceptance)

Networkacceptance

The overall strategy model

is a procedure of filtering

customer requirements,

with increasingly higher test

difficulties and risks from

bottom to top. Tests at

upper layers require morestrict control. The overall

strategy model aims to

finally simplify the site

acceptance through the

filtering at the three lower

layers.



Contents

• Overview


•

FAQs • Negotiation Cases




Q1: Application Stage and Personnel of the Solution

Stage Role Activity

Contract

bidding stage

Engineering bidding

project manager

1. Includes the acceptance standards and test cases completed by the technical

owner based on this solution in the contract.

2. Negotiates on the acceptance standards with customers under assistance of the

technical owner.

3. Tries to include the suggested clauses of the solution (clauses can be selected

according to project situations) in the contract if acceptance standards cannot be

clearly determined with customers at this stage.

Technical owner

1. Specifies strategies according to acceptance items in the solution, and uses test

case selection tools to customize test cases for acceptance as an attachment of the

contract according to the product function range specified in the contract.

2. Helps the engineering bidding project manager to negotiate with customers on test

standards and includes the test cases as an attachment of the contract.

Contractdelivery stage

Engineering delivery

project manager

1. If acceptance standards are not clearly specified in the contract, determines the

acceptance standards with customers at the delivery preparation stage according to

this solution.

2. Determines acceptance standards with customers under assistance of the

technical owner.

Technical owner

1. If acceptance standards are not clearly specified in the contract, specifies relevant

strategies according to the acceptance standards in this solution, and customizes test

cases for acceptance as an attachment of the contract according to the product

function range specified in the contract.

2. Helps the engineering project manager to negotiate with customers.




Q2: How Is This Solution Included in the Contract at

the Pre-Sales Stage?

Responsibility

division interface

Project management

proposal

Engineering bidding documentAcceptance flow

Acceptance milestone

Acceptanceproposal

Construction

period schedule

… …

Engineering service and project

management proposalsAcceptance scope

Acceptance clauses

Acceptance certificate

Acceptance standard/test

case

……

After bidding document negotiation, the acceptance standards or test

cases agreed by both Huawei and the customer are included in the

contract as a part of the acceptance solution.




Q3: How to Use this Solution at the After-Sales Stage?

If test cases are not specified or not clearly specified in the contract,

test standards must be determined with the customer and both

parties must sign for confirmation at the delivery preparation stage

before the acceptance is started.

No acceptance should be performed before acceptance standards

are clearly determined.



Contents

• Overview


• FAQs

• Negotiation Cases




Case 1:

Acceptance steps determined by the

customer of project R in country I:

1. LAB Acceptance

2. FOA Acceptance

3. Installations, Integration and Site

Acceptance

4. RF Cluster Acceptance

5. Market Acceptance

6. SDCA Level Acceptance

The success of the tests in a

previous step is the precondition for

the tests in the next step.

1667 basic test itemsfor LAB/FOA

Results after communication with the customer:

• Nearly half of the test items are delivered in other modes, greatly reducing the on-site test workload

and risks.




Case 2

Test Items Required by the Customer of Project V in Country V: The customer requires testing all features including the OSS and RAN and completing the acceptance

within 30 days after installation.

Risk Analysis The workload to test all features is heavy, and some test conditions are not met on site.

The time schedule is tight, and failure to complete the tasks on time may incur penalty.

Results after Communication with the Customer: Perform feature tests in batches and test the features to be put into commercial use on the existing

network before the PAC is obtained.

Test other features in three months after the PAC is obtained. If test conditions are not met on site,

Huawei can provide internal test reports.

Launch acceptance PAC

Three months

RAN: 160 features

OSS: 133 features

RAN: Internal test reports

can be provided for 154

features



Thank youwww.huawei.com