spirent introduction welcome canada - testforce tests for... · spirent is known for quality and...

Spirent IntroductionWelcome Canada

2Spirent Communications PROPRIETARY AND CONFIDENTIAL

Who is Spirent?

HQ: Crawley, UK Positioning HQ: Paignton, UK

Founded 1936 Three Divisions:

Networks & Applications, Wireless & Service Experience, and Service Assurance

2015 Sales $477.1M

Testforce (Canada) /Sales Support Sales, Sales Engineers and Tech Support* located in North America Testforce has been exclusive distributor for Spirent in Canada since 2002 (14 years)


Who is Spirent (Positioning/Navigation)?Proven and Trusted

30 years experience in positioning technology Spirent is the GPS/GNSS vendor choice of key GNSS

Organizations. Used by GNSS system operators (GPS, Galileo, GLONASS, etc.) Chip set and GPS/GNSS receiver manufacturers GNSS subject matter, academic and defense/military experts

Models created from ground up and tested over 30 years. Proven models mean trusted results. No false positives or incorrect fails reported

due to simulator performance Signal integrity and accuracy consistent throughout product line


Who is Spirent (Positioning/Navigation)??Proven and Trusted

Uncompromised quality and performance at all levels Spirent is known for quality and high performance. We also offer solutions for

new users to GNSS, fundamental testing needs and production test. Quality never compromised – Quality test equipment important at all phase of

test and levels of

Expertise and Information Resource White papers and eBooks available on a variety of topics from how to select a

simulator to running fundamental GNSS tests to application specific testing needs.

New white papers/eBooks released regularly www.spirent.com

http://www.spirent.com/


Certifications and Validations

Spirent simulators have been fully certified by GNSS system operators for nearly 20 years

GPS-JPO security approval for SA/A-S and EVTP testing

GPS Wing security approval for modernised user equipment testing (MUE)

Galileo capability validated by the Galileo Design Authority (ESA)

Spirent has been approved by the Russian Federation for GLONASS Simulators

Additionally Simulation by Spirent simulators has been validated by numerous user segment authorities including:

• RTCM11010.2 testing (Maritime GNSS equipment)

• IEC 61108-3 (Personal Locator Beacons)

• 3GPP (Smartphones)


Navigation/Position GNSS Product Line GNSS Test Equipment Robust PNTAutomationProfessional Services


Navigation/Positioning Solutions

Positioning Test Equipment• GNSS Simulators from A to Z• RF Record and Playback for field testing

Robust PNT • Test for GNSS receiver resiliency • Test equipment and services

Automation• Software to automate ALL test equipment and provide reports• Off the shelf and custom solutions

Professional Services • GNSS simulator and test expertise• Test Scenarios to test plan development to custom services


GSS6300Single Channel 5 constellations

GSS6300M4-8 channels Embedded Controller

GSS670012 channels

Multi software

GSS6425RF Record &

Playback

GSS9000Multi-Frequency

16 channels / 160 Channels

R & DProduction

Navigation/Position GNSS Test Equipment

Field TestVendor Selection/Integration/Validation


Introducing the Spirent Interference Detector

The Spirent Interference Detector is a fixed listening device

that monitors the RF environment in the GPS bands, over a

rangeof approx. 500m, for potential sources of

interference to the GPS signal.


Which Test Methodology is best fit?When to use a simulator vs RF record and playback?

Proving performance requires a Simulator. It is the only way to get known accurate test data

User Experience is also necessary. Recorded real signals best options because is repeatable

Method / Attribute Live-Sky Simulation RPSRepeatable Controllable Partial

Reference truth Realistic Representative


Commercial Applications & Use-Case Examples

•Does your vehicle combine GNSS, CANbus or sensors for positioning?

•Are you drive testing or using repeatable test solution?

•How does your device handle leap seconds?

•How do segment errors or GNSS outages impact performance?

•Hovering/vibration•Landing drone on moving vehicle•Multi-frequency or dual-antenna

•Can you confirm your location is accurate?

•Has your design decreased GNSS receiver performance?

IoT / Wearables Drones

AutomotiveTiming


Defense/Space Applications & Use Case Examples

•Are you able to accurately predict performance at launch, orbit and re-entry?

•Testing GNSS and IMU simultaneously

•Test phase shifting, spectral filtering or adaptive beamforming.

•Radiated/anechoic chamber or conducted

•Test extreme inflight maneuvers in lab before flying

•Add terrain obscurations to test performance

•Jet fighter landing on aircraft carrier (moving object)

•Testing 6 DOF with flight simulator and HIL

Vehicles / aircraft /

autonomousHigh-

dynamics

SpaceCRPA / anti-jamming


Key Tests For Evaluating Performance of GPS/GNSS ModulesCanada - 2016


Agenda

GNSS constellation updates Why is GPS/GNSS testing needed? Identify key basic static tests Testing tools Q&A Summary


GNSS Constellation Update

40 % Receivers are Galileo Ready [1]

Source: GPS World, May 2016


GPS Modernization

Source: US PNT Co-ordination Office, March 2014


GPS Constellation Status @ January 2015 (still the same)

31 Satellites in orbit 30 Operational

• 3 - GPS IIA L1 P(Y)+C/A L2 P(Y)- Launched 90-96

• 12 - GPS IIR- Launched 98-04

• 7 - GPS IIR-M Added - L2C, M-code L1 & L2- Launched 05-09

• 8 - GPS IIF Added - L5- Launched 10- (4 launched in 2014)

1 in Maintenance• GPS IIR-M


GLONASS

Full system in 2010, after Decree No. 587 August 2001

Today 24 satellites in orbit, 24 operational

Recent setbacks & problems July 2013 3 satellites launch failure

5 December 2010 3 satellites launch failure

April 2014 Incorrect ephemerides upload

New-generation Russian GLONASS-K satellite began regular broadcasts on Feb. 15

Additional monitoring facilities planned Nicaragua, AntarcticaSource: http://glonass-iac.ru/en/GLONASS/ & GPS world

http://glonass-iac.ru/en/GLONASS/


Galileo UpdateGalileo Space Segment will include a constellation of a total of 30 Medium Earth Orbit (MEO) satellites, including 6 spares, in a so-called .

Source: http://www.esa.int/Our_Activities/Navigation/The_future_-_Galileo/What_is_Galileo

Experimental phase (two satellites) launched respectively December 2005 and April 2008

In-Orbit Validation (IOV) phase (four satellites) First pair was launched Oct 2011 & next pair in Oct 2012

Full Operational Capability (FOC) phase (four IOV satellites plus 26 FOC satellites) an intermediate initial operational capability (IOC) milestone

with 18 satellites in operation Four pairs of FOC satellites launched by Soyuz from French

Guiana, between Aug 2014 and Dec 2015


BeiDou UpdateBeiDou will include a constellation of a total of 5 GEO, 3 IGSO & 27 Medium Earth Orbit (MEO) satellites, by end of 2020

A total of 16 satellites were launched during in phase 1 as part of regional coverage by end of 2011

Global coverage started in 2015 with the first launch of a new-generation of satellites

The fifth of the new series, the middle-Earth-orbiting (MEO) satellite will join its four predecessors

China launched the 22nd BeiDou satellite into orbit on Tuesday.

Source:http://en.beidou.gov.cn/


Objectives

Analyze and compare the relative tracking performance of various GNSS receivers such as tracking sensitivity, number of tracking channels, etc.

Understand how to evaluate the time a receiver takes to calculate the first position fix and measure the accuracy of the computed position to a true location

Review metrics to consider during the performance evaluation

Test Setup Automation


Why Test: Fair ComparisonData sheets contain all the parameters that help developers select the right manufacturer – but do they give the complete picture?

SatelliteSignals

Correction Sources

Systems

Time to First FixVibration

Reacquisition

Channels

SBAS Accuracy

Drop Shock

Maximum Speed

Update Rate

Altitude Rating

Maximum Altitude

SimultaneousTracked

ChannelsType of

measurements PPS out

Tracking Sensitivity

AcquisitionSensitivity


Classification of tests (What to test)

There is not an standard reference for Standalone GPS testing, but the most common test are:

Time to First Fix Acquisition sensitivity Tracking sensitivity Time to reacquisition Static Position accuracy

There is not a pass/fail criteria for each test

Critical - Test before and after. Must test prior to integration to compare performance post integration


Applications: Which Performance Factors are Most Important?

Mobile Devices

SpaceCommercial Air Travel

Rail Survey

Galileo GLONASS GPS COMPASS

AutomotiveMilitary Applications


Simulation

Control Repeatability Flexibility Efficiency Completeness

RF ConstellationSimulator System

Test environment scenario generation

RF signals commensurate with ICD

NMEA or other

data

?Comparison of receiver calculated

PVT with known simulation data


Metrics used to measure performance

TTF Yield Position Accuracy

Signal Strength


GNSS test categories

ChipsetMeasurements performance

before integration

IntegrationMeasure inside your product

Field TestMeasurement in real-world

NMEAdata

Issue reports Review results


ChipsetMeasurements performance

before integration

IntegrationMeasure inside your product

Field TestMeasurement in real-world

Common GNSS tests

Test 4: Static Position accuracy

Test 3: Tracking sensitivity

Test 2: Acquisition sensitivity

Test 1: TTFF

Test 5: Time to reacquisition


Test 1: Time to First Fix (TTFF)What is it?

Time to First Fix (TTFF) describes the time and process required for a GPS device to acquire enough usable satellite signals and data to obtain a PVT solution.

Cold Start

• Time unknown

• Almanac unknown

• Ephemeris unknown

• Position unknown

Warm Start

• Time known

• Almanac known

• Ephemeris unknown

• Position known (rough)

Hot Start

• Time known

• Almanac known

• Ephemeris known

• Position known (rough)


Test 1: Time to First Fix (TTFF)How to test

Simulator• Static location• Duration: 24 hours• 24 Hours Static

v5-03

Receiver• ColdStart: Reset

receiver’s NVRAM• WarmStart: Acquire

full alamanc & clear ephemeris

• HotStart: Simple receiver restart

What to look for• Extract NMEA data

and compare against truth

• Due to stochastic nature several TTFFs should be obtained


Test 1: Time to First Fix (TTFF)Expected results

TIME: 184946.000 is HHMMSS.SS After performing a cold start to the receiver Monitor the first change in the GPS quality

indicator from 0 to 1 or 2 If there is a change in the previous

indicator. Check for the presence of latitude and longitude in the message

The TTFF is the time of performing a cold or warm start to the first reported Fix (LLA)

1 UTC of Position2 Latitude3 N or S4 Longitude5 E or W6 GPS quality indicator (0 invalid; 1 GPS fix; 2 Diff.GPS fix)7 Number of satellites in use [not those in view]8 Horizontal dilution of position9 Antenna altitude above/below mean sea level (geoid)10 Meters (Antenna height unit)11 Geoidal separation (Diff. between WGS-84 earth ellipsoid and mean geoid 12 Meters (Units of geoidal separation)13 Age in seconds since last update from diff. reference station14 Diff. reference station ID#15 Checksum





Test 1: TTFF


ChipsetMeasurements from factory

IntegrationMeasure in your product

Field TestMeasurement under

real-world


Understand Acquisition & Tracking StagesAcquisition is coarse & tracking is fine

Source: http://what-when-how.com/a-software-defined-gps-and-galileo-receiver/gnss-receiver-operation-overview-gps-and-galileo-receiver/


ACQUISITION SENSITIVITY

Test 2: Acquisition sensitivityWhat is it?

The Acquisition Sensitivity of a GNSS receiver is defined as the minimum signal level that is required to obtain a PVT solution.

-130 dBm

-145 dBm

-160 dBm

RF power

Frequency domain

A Cold Start is performed in each power increment

Usually the increments is performed by steps of 0.5dB or 1dB

Time between tests must be taken in account by the tester


Test 2: Acquisition sensitivityHow to test

Simulator• Static location• Every 30 secs signal

strength increases by 0.5 dB

• User Action File: Acqu_sens.act

Receiver• ColdStart: Reset

receiver’s NVRAM• WarmStart: Acquire

full alamanc & clear ephemeris

• HotStart: is the same as reacquisition test

What to look for• Observe CN0s

reported• AGNSS receivers

can acquire below -150 dBm


Test 2: Acquisition sensitivityExpected results

Perform a cold start as many times you Decrease the power Decrease the power from the nominal

power level dB by dB Check for SATID and SNR of each satellite

reported Once the SNR of the satellites reported is

unstable you reach its acquisition sensitivity

1 Total number of messages of this type in this cycle

2 Message number

3 Total number of SVs in view

4 SV PRN number

5 Elevation in degrees, 90 maximum

6 Azimuth, degrees from true north, 000 to 359

7 SNR, 00-99 dB (null when not tracking)

8 - 11 Information about second SV, same as field 4-7

12 - 15 Information about third SV, same as field 4-7

16 - 19 Information about fourth SV, same as field 4-7


Test 4: Position accuracy



Test 1: TTFF





real-world


Test 3: Tracking sensitivityWhat is it?

The minimum satellite power level where the receiver can keep tracking the code and carrier phase and maintain a position fix.

TRACKING SENSITIVITY

-130 dBm

-145 dBm

-160 dBm

RF power

Frequency domain

Single Cold Start performed in the beginning of the test

Time between tests must be taken in account by the tester

A statistical analysis is required. Several test need to be performed by the tester


Test 3: Tracking sensitivityHow to test

Simulator• Static location• Every 30 secs signal

strength decreases by 5/0.5 dB

• User Action File: track_sens.act

Receiver• Receiver is in

continuous tracking mode – no changes needed during the course of this test

What to look for• Observe CN0s

reported• Receivers continue to

track satellites below acquisition sensitivity


Perform a cold start as many times you Decrease the power Decrease the power from the nominal

power level dB by dB Check for SATID and SNR of each satellite

reported Once the SNR of the satellites reported is

unstable you reach its acquisition sensitivity

1 Total number of messages of this type in this cycle

2 Message number

3 Total number of SVs in view

4 SV PRN number

5 Elevation in degrees, 90 maximum

6 Azimuth, degrees from true north, 000 to 359

7 SNR, 00-99 dB (null when not tracking)

8 - 11 Information about second SV, same as field 4-7

12 - 15 Information about third SV, same as field 4-7

16 - 19 Information about fourth SV, same as field 4-7

Test 3: Tracking sensitivityExpected results





Test 1: TTFF





real-world


Test 4: Static Position accuracy What is it?

The purpose of this Key Performance Indicator is to test how accurately the receiver can determine its true position.

Cold Start Warm Start Hot Start

Picture reference: http://blog.oplopanax.ca/2012/11/calculating-gps-accuracy/


Test 4: Static Position accuracyHow to test

Simulator• Any static scenario

can be used

Receiver• Receiver is in lock

mode – no changes needed during the course of this test

What to look for• Several measures to

position accuracy: CEP, SEP, 3DRMS, 67% error & 95% error


Test 4: Static Position accuracyExpected results

TIME: 184946.000 is HHMMSS.SS LATITUDE DDMMM.MMMMM => D+MMM.MMMMM/60LONGITUDE DDDMM.MMMMM =>DDD+MM.MMMM/60 Considerate if it is N/S or W/E sign

dependency A known position (LLA) Calculate the distance between the known

position to the reported position. Typically it is known as horizontal/vertical error



Dimensions Accuracy Measure

Probability (%)

Typical Usage

1 rms 68 Vertical

2 CEP 50 Horizontal

2 rms 63-68 Horizontal

2 R95 95 Horizontal

3 2drms 95-98 Horizontal

3 rms 61-68 3-D

3 SEP 50 3-D

Test 4: Static Position accuracyAccuracy Measures

Ref: ArtigoAcuraciaGPSsemAutor.pdfhttp://gpsworld.com/gps-accuracy-lies-damn-lies-and-statistics/


Test 4: Static Position Accuracy

Test 3: Tracking Sensitivity

Test 2: Acquisition Sensitivity

Test 1: TTFF





real-world


Test 5: Time to reacquisitionWhat is it?

Re-acquisition time is the time necessary for a receiver to regain a PVT solution after total loss of all received signals

Outage simulated


How to testTest 5: Time to reacquisition

Simulator• Any scenario• On/off commands to

control duration of signal loss

• User Action File: re-acqu_time.act

Receiver• Receiver is in

continuous tracking mode – no changes needed during the course of this test

What to look for• Observe time to fix

and CN0s • Satellite geometry is

a factor for this test


Test 5: Time to reacquisitionExpected results

TIME: 184946.000 is HHMMSS.SS Monitor the change in the GPS quality

indicator from 0 to 1 If there is a change in the previous indicator.

Check for the presence of latitude and longitude in the message

The TTR is the time that the receiver takes to reacquire and lock positon when and outage happened

Time to Reacquisition is faster than TTFF



R&D Integration Verification Production

GSS9000 GSS6700 GSS6425 GSS6300 (M)

Testing toolsSpirent simulators and RPS

Processes Application examples


PT TestBenchSpirent’s PT TestBench is an automation and reporting software tool, which provides you with an integrated test solution, enabling characterisation and vulnerability assessment of GNSS receivers using pre-defined Test Cases.


PT TestBench：User Interface

Use chart or table to show the key data extracted from the test, store the

data into external database for later usage

编辑

iTest Activities streamline testing efforts around the

most common automation tasks

Quickly add automated test steps through

capture and analysis. Easily execute your test and view test

reports or share result with others

Create & edit test cases in an easy to edit format

that doesn’t requirement scripting

Add analysis to create robust test cases. Responses can be abstracted to allow test case portability and ease of

maintenance


Sample Reports


Time to start thinking about Vulnerabilities?

Data from Broadcom Corporation


Q & A


SummaryWhat did we learn today?

Understand your application and GNSS related use cases There are no defined standards for GPS/GNSS testing,

but there are common tests effective for defining performance Understand the results For more information visit:

www.spirent.com Follow us on LinkedIn : Spirent Communications / Spirent Positioning and Navigation

Please provide feedback and complete brief survey Contact us for more info: [email protected] [email protected]

http://www.spirent.com/

mailto:[email protected]

spirent introduction welcome canada - testforce tests for... · spirent is known for quality and...

Documents