October 2015 AllSeen Alliance 1

Interoperability Promise

Jose Aurelio Rodrigo20 October 2015, AT4 wireless

October 2015 AllSeen Alliance 22

1. AT4 wireless2. Interoperability Promise3. Success Cases4. How Interoperability can be ensured?

Agenda

3

AT4 wireless

October 2015 AllSeen Alliance 4

• Telecommunications • Automotive/Connected Car• Safety• Medical & Health• Smart Home/Smart Grid• Smart Cities• M2M/IoT• Defense/Military • Consumer Electronics• Appliances• Aerospace, Naval & Railway• Game & Slot Machines• Green Energies• Engineering

Certification Services

Laboratory Services

Carrier Services

Test Engineering

and R&D

AT4 wireless

Mobile GSM/GPRS/EDGE/AMR UMTS HSPA+ SIM/USIM LTEWireless Wi-Fi™ Bluetooth® AllSeen Alliance Wireless Power NFC/RFID EMV ® ULE Mobile Payment Mirrorlink (Connected Car) Continua (Health)Internet of Things (IoT)

EMC RF Over-The-Air (OTA) SAR Audio/Acoustics Automotive/Connected Car Defense/Military Medical

Industry / Regulatory Batteries Smarthome Smartcities Photovoltaic Lighting Electrical Safety Environmental Calibration

Communications Management

Systems

October 2015 AllSeen Alliance 5

more than 100 Worldwide Partners

and over 6.500 Compliance

Certificates achieved

AT4 wireless, Inc.(Taiwan)

AT4 wireless, S.A.U(Spain)

Locations Agents - Consultants Compliance Partners

AT4 wireless JapanAuthorized Test

Laboratory(Partnership with Toyo

Corporation)

AT4 wireless, Inc. (USA)

AT4 wireless Locations

6

Interoperability Promise

October 2015 AllSeen Alliance 7

IntroductionInteroperability is the property of a product or system, to work with other products or systems, present or future, without any restricted access or implementation

The interoperability perception of the users will decide the success of a communication technology

October 2015 AllSeen Alliance 8

A device from one vendor has to operate with other vendors’ devices.The end-to-end behavior of a device in actual operating conditions is as specified on real world systems.The performance of a device is as specified when working in the real world.Devices have to have the ability to recover from communication errors.

The communication between devices is secure.

Interoperability Expected

October 2015 AllSeen Alliance 9

Poor Interoperability is Expensive• In the past, interoperability failures meant:

– Bad publicity in trade magazines– Embarrassment for the manufacturer– Annoyance of the end customer

• Today, interoperability failures in the field means:– Front page headlines – Fall in manufacturers’ stock price– Loss of investor confidence– Unrecoverable damage to brand name– Irretrievable loss of customers– Customers give up and try with another technology

October 2015 AllSeen Alliance 10

Interoperability PerceptionFactors affecting the interoperability perception:

The specification has to be unique,

clear and unambiguous

The implementation has to be correct

and without defects

The connection of a device with other devices has to be simple and easy

October 2015 AllSeen Alliance 11

Wi-Fi Alliance• Many different devices have to interoperate with many different Access Points

• Clear strategy selecting the alternatives and options of the standard: unique path • Testing strongly focused on interoperability only• Very easy way to connect the device with the access point

Keys of Success

• Devices shipped: ≅ 10 Billion• Devices in use: ≅ 4.5 Billion

Facts

October 2015 AllSeen Alliance 12

Bluetooth SIG• Many different devices have to interoperate with many different devices

• Unique standard with evolutions. Fast reacting against interoperability issues• Testing balanced between RF & Protocol conformance and interoperability• Some intervention of user required to pair devices

Keys of Success

• Devices shipped: ≅ 2.5 BillionFacts

October 2015 AllSeen Alliance 13

Cellular (3GPP): GCF and PTCRB• Many different devices have to interoperate with some different base stations

• Unique standard per generation with evolutions. International consensus• Testing focused on RF & Protocol conformance with interoperability (field testing)• Very easy way to connect the device with the network

• Devices shipped: ≅ 14 Billion• Devices in use: ≅ 6.6 Billion

Facts

Keys of Success

October 2015 AllSeen Alliance 14

Non Predominant Technologies• Many different devices have to interoperate with many different devices

• No unique standard or standard with several branches without consensus• Testing focused on conformance without interoperability• Difficulty for the user to identify the devices that are compatible with his device

Keys of Current Situation

• The technology is not adopted by users and the is finally abandoned

Facts

October 2015 AllSeen Alliance 15

What About AllSeen Alliance?• Many different devices have to interoperate with many different devices

• Unique standard without branches: only one path• Testing focused on a balance between conformance and interoperability• Devices should find the way to acknowledge and interconnect between them

Keys of Success?

October 2015 AllSeen Alliance 16

How interoperability can be ensured?

Compliance & Certification ProgramThe purpose of this program is to certify a product’s compliance with the requirements of AllJoyn. Certification ensures that a product:• Conforms to the AllJoyn Interface Definitions and• Maintains Interoperability with other AllJoyn Certified compliant products.

Conformance Testing+

Interoperability Testing

By means of establishing a certification program with testing

October 2015 AllSeen Alliance 17

Certification Testing: Conformance

• Testing performed using a device emulation tool to verify feature by feature the compliance of an AllJoyn implementation with AllJoyn interface definitions

AllJoynImplementation

under Test

Emulation Tool: Certification Test Tool

October 2015 AllSeen Alliance 18

Is Conformance Testing enough?

But, will a product that has successfully passed Conformance Testing be able to smoothly interoperate with other certified AllJoyn products?

October 2015 AllSeen Alliance 19

Is Conformance Testing enough?Devices passing Conformance Testing as defined in Test Specifications have a higher likelihood of interoperability, but it is not guaranteed:

• Conformance testing checks independently each step of the supported functionalities. But a complete test of the whole functionality is not performed.

• A few different conformance scenarios are tested at step level. But more different scenarios covering the whole functionality are necessary.

• Certification Test Tool is an official correct realization of AllJoyn code. But testing with other conformant realizations of AllJoyn code is necessary.

Test Case 1

Test Case 2

Test Case 3

Test Case 4

October 2015 AllSeen Alliance 20

The lack of interoperability can be caused by different reasons not discoverable in Conformance Testing:

Is Conformance Testing enough?

• Interface Definition completeness• Different interpretation of Interface Definitions among

OEMs• Different configuration used among devices (parameters

using different values allowed by the spec), leading to untested scenarios

• Different Timing implementations• Ability to recognize and handle data coming from other

devices

October 2015 AllSeen Alliance 21

Causes of Interoperability Problems• Interface Definitions & Developer Guideso Errors and ambiguities in Interface Definitions and Developer Guides

• Implementationso Human errors, e.g. programmer errors when integrating the AllJoyn frameworko Modifications to the AllJoyn frameworko Different interpretations of the Interface Definitions and Developer Guideso Different choice of options allowed by the Developer Guides

• Technologyo Wi-Fi networks might use different traffic transport techniqueso Physical device compatibility problemso Physical device configuration problems

October 2015 AllSeen Alliance 22

Conformance testing needs to be complemented with Interoperability Testing to ensure that AllJoyn devices can communicate and interwork with any other AllJoyndevices

• Interoperability Testing verifies if the end-to-end functionality between, at least, two AllJoyn devices in communication is as required by AllJoyn specifications

• Conformance Testing improves the chances of interoperability while interoperability Testing checks at user level if interoperability has been achieved

Interoperability Testing

AllJoynImplementation

under Test

Auxiliary AllJoyn

Implementation

October 2015 AllSeen Alliance 23

Interoperability Testing• Shows that two devices or more interoperate within a limited scenario• Tests at a ‘high’ level (as perceived by users)

– Tests the ‘whole’, not the parts e.g., AllJoyn Core + services

• Interoperability Testing is BROAD and SHALLOW– Less thorough but wide in scope– Gives a high-level of confidence that devices (or components in a system) will

interoperate with other devices (components)

The specification has to be unique, clear and

unambiguous

The implementation has to be correct and

without defects

The connection of a device with other devices has to be simple and easy

October 2015 AllSeen Alliance 24

The basic requirements for Interoperability Testing are:

Interoperability Testing Implementation

• To ensure that an AllJoyn device can operate with any other AllJoyn device

&

• To be affordable. It is not possible to test with ALL other AllJoyn devices in the market

A trade-off is necessary to have the highest

probability of interoperability with a

reasonable use of resources (in time and

cost)

October 2015 AllSeen Alliance 25

Interoperability Testing Implementation

• A method for determining to what extent two or more AllJoyn implementations function together for some range of features, services and use cases over specific scenarios

• Interoperability testing has to be performed assessing the end-to-end services operation across two or more products

• But, interoperability testing shall focus on interoperable interfaces, not on internal functional behavior

• More assurance of interoperability of a device is achieved by means of:o Testing as many pairings of this device with other devices as possibleo Testing pairings of this device with devices from different OEMs (avoiding the

repetition of OEMs)

October 2015 AllSeen Alliance 26

Limitations of Interoperability Testing• Does not prove interoperability with other implementations with which no testing

has been done– A may interoperate with B and B may interoperate with C. But it doesn’t necessarily

follow that A will interoperate with C.

• Does not prove that a device is conformant– Interoperable devices may still interoperate even though they are non-conformant

• Cannot explicitly test error behavior or unusual scenarios– Or other conditions that may need to be forced (lack of controllability)– Has limited coverage (does not fully exercise the device)

• Not usually automated and may not be repeatable

October 2015 AllSeen Alliance 27

• For more information on AllSeen Alliance, visit us at: allseenalliance.org & allseenalliance.org/news/blogs