oneM2M Testing Activities

Prof. Song JaeSeungSejong University & KETIoneM2M TST WG Chair

jssong@sejong.ac.kr

WoT OpenDay (2)

Outlinen Interop & Conformance Testingn oneM2M Testing Activitiesn oneM2M Interop. Eventsn oneM2M Certification Program

WoT OpenDay (3)

n Tests (end-to-end) functionality between 2 or more products

n It shows, from the user's viewpoint, that functionality is accomplished (but not how)

n Scenario-based system testing

Interoperability testing

WoT OpenDay (4)

n Conformance testing concentrates on specific components in a system

n Conformance testing is applied over open interfaces and checks for conformance to the requirements in a base specification

n Unit testing

Conformance testing

WoT OpenDay (5)

n Both are complementary!n Product could happen to be conformant but not

interoperablen Interop testing is more appropriate when the standard

is in development phase . It helps to validate the standards

n Conformance testing is more appropriate for testing stable specifications and is for testing product and check if the standard is correctly implemented

Interoperability & Conformance testing

WoT OpenDay (6)

n Methodology: » TS-0015 : Testing Framework

n Interoperability Testing » TS-0013: Interop Testing

n Conformance Testing » Conformance on oneM2M primitives: PICS TS-0017,

TSS&TP TS-0018 and ATS TS-0019 (oneM2M TF-001 & ETSI STF 531)

» Security conformance: PICS TS-0027, TSS&TP TS-0028 and ATS TS-0029

n Definition of product profiles : TS-0025 n Developer Guides : Series of eight technical reports

oneM2M Testing Specifications

WoT OpenDay (7)

n Interoperability testing specification» R1 TS-0013 Interoperability testing à Stable and used in Interop events» R2 TS-0013 Interoperability testing à Under development with new

featuresn Release 2 Interop spec. features

» Semantic testing» Security testing» Interworking testing» Data model testing

n Two oneM2M Interop. Testing events every year

oneM2M Interoperability Testing

WoT OpenDay (8)

n Conformance Testing specifications » TS-0017, Implementation Conformance Statement (ICS)» TS-0018, Test Suite Structure & Test Purposes» TS-0019, ATS (TTCN-3)» TS-0025, Product profiles

n TTCN-3 test cases are developed for all R1 product profiles» oneM2M TF-001 and ETSI STF-531 collaborate to deliver TTCN-3 test

cases» Validation of the test cases with 2~3 implementations and 2~3 oneM2M

testing tools» Now all test cases are validated and the test cases are being used by

oneM2M CB and Testing Lab for testing» Interop events are being used to validate the test cases and tools

n Now Rel-2 & Rel-3 test cases are being developed

oneM2M Conformance Testing

WoT OpenDay (9)

TST

Features catalogue & Product profiles

• product profiles and feature catalogue– guidebook to my oneM2M product features – fills the gap btw. function specs. and test specs.

FeaturesCatalogue(TS-0031)

FunctionalArchitecture(TS-0001)

ProductProfiles(TS-0025)

glimpseofoneM2M

summarized

guidelineforproductplanning

preparetestcasesforproducts

referred

CoreProtocol(TS-0004)

TestPurposes(TS-0018)

AbstractTestSuite(TS-0019)

whichfeaturetobeimplemented?

whichfeatureneedstobetested?

WoT OpenDay (10)

Product profiles

• Rel-1 profiles define in total seven products

Profile DescriptionADN Profile 1 Constrained sensor as ADNADN Profile 2 Constrained actuator as ADNADN Profile 3 Normal sensor devicesADN Profile 4 Small originator device types of oneM2M servicesIN Profile 1 Server device type of oneM2M servicesASN Profile 1 Constrained actuator as ASNMN Profile 1 Gateway devices that support multiple different area

network technologies and connect devices

WoT OpenDay (11)

Developer guide series

n example scenarios and binding messages

Deliverable Title Examplesof

TR-0025 Applicationdeveloperguide HTTPbinding,XML/JSONserialization

TR-0034 TemperaturemonitoringexampleusingCoAPbinding

CoAPbinding,<pollingChannel>

TR-0035 DeveloperguideofDeviceManagement <mgmtObj>,OMADM,LwM2M,BBFTR-069

TR-0037 SmartFarmExampleusingMQTTBinding MQTTbinding

TR-0038 Developerguide- Implementingsecurityexample

Provisioning,SecurityAssociationEstablishment

TR-0039 Developerguide- SDTbasedimplementation SDTforhomeappliances

TR-0045 DeveloperGuide:ImplementingSemantics Semanticannotationanddiscovery

TR-0048 DeveloperGuideof3GPPInterworking (TBD)

WoT OpenDay (12)

n Testing and Test Control Notation Version 3 n Internationally standardized language developed specifically

for executable test specification» Specified by ETSI MTS Technical Committee » Is independent of a specific IUT or IUT interfaces » Is independent of a test execution environment » Standard available at portal.etsi.org via ETSI programme

n Allows unambiguous implementation of tests n Look and feel of a regular programming language n Good tool support (many commercial tools available) n Successfully deployed in different organizations and industry in

a variety of application domains » e.g., telecom, automotive, software, etc.

What is TTCN-3?

WoT OpenDay (13)

n Validation reveals problems/errors in » Standards and Products

n Validated standards give a higher chance of interoperable products » For standardisers gives assurance that they provide right functionality » For manufacturers and operators gives confidence to implement and go

to market n Provides an opportunity to correct errors in a controlled

manner » Late fixes in the product cycle are more expensive than early ones » Decreases time to market

Why Validate Standards

Standards can be validated by several means but one of the most practical and cost effective is by interop events

WoT OpenDay (14)

n Co-organized and funded by TTA and ETSIn Twice a year n Free of charge n Open to all companies with oneM2M implementations

(members and non-members) n Covered by NDA. No companies results are published n Important technical feedback provided to oneM2M

oneM2M Interop at a Glance

1st 2nd 3rd 4th

Regularevents

Verticals

2015 2016 2017

5th

SemanticInterop

2018

6th

WoT OpenDay (15)

oneM2M Interop #1 ~ #4Interop#1 Sophia-Antipolis (2015) Interop#2 Seoul (2016)

Interop#3 Kobe (2016) Interop#4 Taipei (2017)

WoT OpenDay (16)

n Co-organized by TTA and ETSIn When: 4th Dec (Mon) to 8th Dec (Fri) 2017, 5 daysn Where: Global IoT Certification Center, in Pangyo, Korean Scope:

» Interoperability and conformance testing (TS-0013, TS-0018, TS-0019)» Release 1 & 2 (TS-0001 & TS-0004 & Binding baselines from TP #31)

n Conformance Testing is the core of Interop5» In 5th Interop Event, Conformance session will be expanded to

• Validate Test Purpose, TTCN-3 (oneM2M ATS)• Validate Test Systems• Run Conformance Test on participants’ implementations• Semantic testing

» Semantic testing (an half-day)

oneM2M Interop #5

WoT OpenDay (17)

oneM2M Interop #5 – Semantic Testingn 4 companies implemented CSE or AE capabilitiesn 1 Four (4) hour session

» Tests developed based on Developer Guide• <semanticDescriptor>

• Semantic Discovery

n Information Session» Overview of oneM2M Semantic Features

n Planned topics for next Semantic Interop» Semantic Validation» Semantic Query» Semantic ACP» Partial Update of <semanticDescription>

WoT OpenDay (18)

oneM2M Semantics Features

• SemanticsannotationinR2(via<semanticDescriptor>)providesabasicsemanticsfunction,whichenablesothersemanticfeatures(e.g.semanticsresourcediscovery,semanticsquery,etc.)

• OntologymanagementinR3enablestoflexiblyleverageotherontologiesforsemanticsfeatures

• Moreadvancedsemanticsfeaturessuchassemanticsreasoningandsemanticsanalytics,whichhavebeenidentifiedasexistingrequirementsandcouldbespecifiedinR4.

oneM2MR2 oneM2MR3 oneM2MR4

• SemanticsAnnotation• SemanticsResource

Discovery• PartialTripleUpdate

usingSPARQL

• SemanticsQuery• SemanticsMashup• SemanticsACP• SemanticsValidation• OntologyManagement

• SemanticsReasoning• SemanticsAnalytics• …

WoT OpenDay (19)

oneM2M Semantics Features

WoT OpenDay (20)

Semantic Interop Scenario

Sensor#1IndoorTemperature

Fahrenheit

Sensor#2OutdoorTemperature

Celsius

Server

Smartphone#1Sensor#3

IndoorTemperatureCelsius

Sensor#4FridgeTemperature

Celsius

ADN-AE-2

ADN-AE-1

ADN-AE-3

ADN-AE-4

IN-CSE

ADN-AE-5

Mca

Mca

Mca

Mca

Mca

Jena

server

tempsensorae2

tempsensorae1

temperature

CI1

CI2

SD1

CIN

tempsensorae3

tempsensorae4

WoT OpenDay (21)

Semantic Interop Scenario

• CreateSDforAE1-4.• SemanticDiscovery

1. Naturallanguagequery:“GivemeallthesensorrelatedresourcesthatmeasurethetemperatureintheunitCelsius.”

2. Naturallanguagequery:“Givemeallthesensorrelatedresourcesthatmeasuretheindoortemperatureofahouse.”

3. Naturallanguagequery:“GivemeallthesensorrelatedresourcesthatmeasuretheindoortemperatureofahouseandprovidethemeasurementintheunitFahrenheit.”

oneM2M Certification Program

WoT OpenDay (23)

oneM2M Certification program

• TTA was agreed the first oneM2M Certification Body in SC #33(Sep. 2016).

• oneM2M Certification Program was officially launched at Feb. 9, 2017.

• oneM2M Certification is intended to create an ecosystem of certified products that ensures interoperability among certified products.

www.onem2mcert.com

WoT OpenDay (24)

oneM2M CB Organization

SC

TP

TST

Certi. Ad-Hoc

oneM2M

oneM2M Certification Body (CB)

AuthorizedTestLab(ATL)

1st oneM2M Certification Body!

oneM2M - oneM2M CB

WoT OpenDay (25)

oneM2M Certified Products

Product Webpage Vendor ProductTypeGWP http://www.irexnet.co.kr IREXNET Endproduct(IN-CSE)

AiSOP http://www.irexnet.co.kr IREXNET Endproduct(IN-CSE)

Insator™ https://www.samsungsds.com SAMSUNGSDS Endproduct(IN-CSE)

HANDYPIAIoT Platform http://www.handysoft.co.kr/ HANDYSOFT,Inc. Endproduct(IN-CSE)

IoT HealthcarePlatform http://www.hconnect.co.kr/ HealthConnectCo.,Ltd Endproduct

ThingPlug https://www.thingplug.net SKTelecom Softwarecomponent

N-MAS http://www.ntels.com nTels Endproduct

IoTMakers Middleware http://iotmakers.olleh.com KT Softwarecomponent

IoTMakers http://iotmakers.olleh.com KT Softwarecomponent

e-IoT EnergyPlatform https://spin.kepco.co.kr KEPCO Endproduct

e-IoT EnergyGateway https://spin.kepco.co.kr KEPCO Endproduct

WoT OpenDay (26)

Global oneM2M Certification Program• The current certification program operated by TTA is now being

migrated to GCF for global program

• Industry advisor group is activated for this purpose• 1Q 2019 for Rel-2 and Rel-3

oneM2M Testing Tools

WoT OpenDay (28)

IoT Packet Analysern Simple user interface and operation process

• Packet Analysis - Show the packet structure of uploaded pcap file.

• Packet Compare - Same as above. Only the process of loading sample packets stored in the server is added.

Packet Analysis

Packet Compare

Web page

click

Click the button you want to perform

Browse…

File upload

Select the pcap file to upload from your computer

0000.packet

Select the packet you want to analyze

0001.packet0002.packet0003.packet0004.packet

….…. ok

A result page showing the structure of the packet.

All

oneM2M….…. ok

Select filter

WoT OpenDay (29)

n oneM2M web-based tester Introduction page

oneM2MTester

WoT OpenDay (30)

Ttworkbench and TTCN-3 for oneM2M

oneM2M Functional Architecture – TTworkbench Test Access

FieldDomain

Infrastructure Domain #1

Application (AE)

Network (NSE)

Mcc

Mca

Mcn

Mca

Common Service (CSE)

Application (AE)

Network (NSE)

Mca

Mcn

Common Service (CSE)

Mca

Mcc Mcc

Optionally available via non-standard test API Standard Test API

Infrastructure Domain

Provider #2

Sensors, Actuators, Gateways

TTworkbench TTCN-3 Conformance test tool

WoT OpenDay (31)

Ontology Validatorn Developed by EGMn Basic aspects to be validated on semantic information

» Lexical validation» Syntactic validation» Semantic validation

Format FilenameextensionN-Quads .nqN-Triples .ntN3 .n3Turtle .ttlTriG .trigTriX .trixRDF/JSON .rjJSON-LD .jsonldBinaryRDF .brfRDF/XML .rdfOWL .owl

Popularreferenceontologies

areintegrated in

thetool

WoT OpenDay (32)

F-Interop H2020 Projectq www.f-interop.euq 1 November 2015 – 31 October 2018q develop and provide online interoperability and

performance test tools to support emerging technologies from research to standardization and market launch

q InitiallystandardsoftheIoT realmq Take,asastartingpoint,theETSIplugtests specificationsand

buildanarchitecturethatallowsthosetobedoneremotely

• CoAP• oneM2M• 6TiSCH

WoT OpenDay (33)

q oneM2M Testing Activitiesq oneM2M Interop. Eventsq oneM2M Certification Programq oneM2M Testing Tools

Summary

THANK YOUProf. Song JaeSeungjsong@sejong.ac.krSejong University & KETI