[american institute of aeronautics and astronautics 2007 u.s. air force t&e days - destin,...

13
American Institute of Aeronautics and Astronautics 1 Common Range Integrated Instrumentation System (CRIIS): Meeting the Future “TSPI Truth Source” Needs of the T&E Community Sultan Mahmood 1 and Jeannine Yakaboski. 2 Jacobs Engineering (TEAS), Eglin AFB, Fl.,32542 This paper will discuss the accuracy requirements of the next generation of Time Space Position Information (TSPI) equipment, CRIIS, to be designed to serve as the golden “Truth Standard” for the Tri-Service Test and Evaluation (T&E) community. The equipment to be procured can be grouped into a number of configurations characterized by various levels of TSPI accuracies, data link and encryption capabilities. The feasibility of developing truth standards for the already highly capable test items is critically evaluated under the Technology Readiness Level (TRL) guidelines of DoD 5000.2-R document. The rapidly improving GPS signals and receiver technologies, coupled with sophisticated measurement processing and GPS/INS integration methodologies are reviewed for maturity, robustness and practicality. A sample of generic Tri-Service User requirements and CRIIS specifications are examined with respect to the heretofore common rule-of-thumb standard, ”Truth Source Must be 10 Times More Accurate Than The System Being Tested”. A more practical acceptance criteria based on statistical data reduction procedures is proposed for CRIIS equipment procurement. Since the performance specifications of CRIIS are extremely stringent, finding a “truth source” to test CRIIS is a great challenge in itself. A comprehensive Hardware-In-the-Loop (HIL) test concept utilizing GPS RF Signal Simulators is developed, and is recommended for use along with the new statistical data reduction procedure. This HIL concept will provide a test-bed for thoroughly testing the requirements, reducing cost/schedule risk and will allow for an “apples-to-apples” comparison between candidate systems. I. Introduction RIIS will replace the Advanced Range Data System (ARDS) that was developed in the 1980s and has been the TSPI standard for over a decade. Currently, most of the US military T&E ranges use the ARDS, acquired by the Range Application Joint Program Office (RAJPO), using funding provided by the Central Test and Evaluation Investment Program (CTEIP) office. Periodic technology insertions and upgrades have continued to be made to the original ARDS equipment, in order to keep up with the growing accuracy and data needs of the users. However, these improvements have lagged the rapid pace of GPS technology and civil/military applications. This paper will discuss the technical capabilities and shortfalls of the ARDS system, emphasizing the need for a complete replacement of the system with new and improved technologies that are either available now or are in the process of development. Several areas of enhancements are identified leading to the documentation and requirement generation for the future TSPI system, CRIIS. The need for modular and open architecture based systems offering flexibility, upgradeability, interoperability is discussed. The challenges of providing continuously increasing accuracy levels for the future T&E ranges, and the activities that are required to develop and test these stringent specifications are analyzed. The three levels of TSPI accuracy with separate real-time and post-mission requirements as established by the CRIIS Program and T&E users are discussed in the context of current state-of-the-art and in- 1 Technical Fellow, GPS/INS CoP, TEAS Contract, 308 West D Ave, Eglin AFB, Fl. 32542 2 Principal Engineer, TEAS Contract, 308 West D Ave, Eglin AFB, FL. 32542 C U.S. Air Force T&E Days 13 - 15 February 2007, Destin, Florida AIAA 2007-1668 Copyright © 2007 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

Upload: jeannine

Post on 16-Dec-2016

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics1

Common Range Integrated Instrumentation System(CRIIS):

Meeting the Future “TSPI Truth Source” Needs of theT&E Community

Sultan Mahmood1 and Jeannine Yakaboski.2

Jacobs Engineering (TEAS), Eglin AFB, Fl.,32542

This paper will discuss the accuracy requirements of the next generation of Time SpacePosition Information (TSPI) equipment, CRIIS, to be designed to serve as the golden “TruthStandard” for the Tri-Service Test and Evaluation (T&E) community. The equipment to beprocured can be grouped into a number of configurations characterized by various levels ofTSPI accuracies, data link and encryption capabilities. The feasibility of developing truthstandards for the already highly capable test items is critically evaluated under theTechnology Readiness Level (TRL) guidelines of DoD 5000.2-R document. The rapidlyimproving GPS signals and receiver technologies, coupled with sophisticated measurementprocessing and GPS/INS integration methodologies are reviewed for maturity, robustnessand practicality. A sample of generic Tri-Service User requirements and CRIISspecifications are examined with respect to the heretofore common rule-of-thumb standard,”Truth Source Must be 10 Times More Accurate Than The System Being Tested”. A morepractical acceptance criteria based on statistical data reduction procedures is proposed forCRIIS equipment procurement. Since the performance specifications of CRIIS areextremely stringent, finding a “truth source” to test CRIIS is a great challenge in itself. Acomprehensive Hardware-In-the-Loop (HIL) test concept utilizing GPS RF SignalSimulators is developed, and is recommended for use along with the new statistical datareduction procedure. This HIL concept will provide a test-bed for thoroughly testing therequirements, reducing cost/schedule risk and will allow for an “apples-to-apples”comparison between candidate systems.

I. Introduction

RIIS will replace the Advanced Range Data System (ARDS) that was developed in the 1980s and has been theTSPI standard for over a decade. Currently, most of the US military T&E ranges use the ARDS, acquired by

the Range Application Joint Program Office (RAJPO), using funding provided by the Central Test and EvaluationInvestment Program (CTEIP) office. Periodic technology insertions and upgrades have continued to be made to theoriginal ARDS equipment, in order to keep up with the growing accuracy and data needs of the users. However,these improvements have lagged the rapid pace of GPS technology and civil/military applications.

This paper will discuss the technical capabilities and shortfalls of the ARDS system, emphasizing the need for acomplete replacement of the system with new and improved technologies that are either available now or are in theprocess of development. Several areas of enhancements are identified leading to the documentation and requirementgeneration for the future TSPI system, CRIIS. The need for modular and open architecture based systems offeringflexibility, upgradeability, interoperability is discussed. The challenges of providing continuously increasingaccuracy levels for the future T&E ranges, and the activities that are required to develop and test these stringentspecifications are analyzed. The three levels of TSPI accuracy with separate real-time and post-mission requirementsas established by the CRIIS Program and T&E users are discussed in the context of current state-of-the-art and in-

1 Technical Fellow, GPS/INS CoP, TEAS Contract, 308 West D Ave, Eglin AFB, Fl. 325422 Principal Engineer, TEAS Contract, 308 West D Ave, Eglin AFB, FL. 32542

C

U.S. Air Force T&E Days13 - 15 February 2007, Destin, Florida

AIAA 2007-1668

Copyright © 2007 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

Page 2: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics2

development GPS/INS technologies. The purpose is to assign risk levels to the acquisition of CRIIS capabilities andmodules, using the DoD 5000.2-R technology readiness level standards and guidelines. GPS signal modernizationand advanced signal processing aspects are addressed, since they may be potentially required to meet the CRIISgoals. The commonly used 10X standard for the truth source is critically examined, in the perspective of newrequirements and limitations of technology. Statistical derivation of a candidate criteria for accepting CRIIS (or anyother truth source) is reported, along with the procedure of applying it even when the 10X rule is not satisfied.Another concept developed for the CRIIS program is the use of Hardware-In-the-Loop simulation for applicationduring the accuracy verification phase. The HIL approach promises to make the task of procuring CRIIS efficientand effective both in terms of capability/schedule and cost optimization, as well as to allow a fair comparison of thecandidate systems.

II. The ARDS GPS-Based TSPI System

Currently, most of the US military T&E ranges use the Advanced Range Data System (ARDS), acquired by theRange Application Joint Program Office (RAJPO), using funding provided by the Central Test and EvaluationInvestment Program (CTEIP) office. The ARDS baseline consists of participant and ground subsystems designedwith built-in interoperability features to allow the tri-service T&E ranges to share the instrumentation assets anddata. Developed in the 1980s, and deployed in the 1990s, the ARDS is a GPS-based TSPI suite of equipmentavailable in pod and internal mount (IM) configurations. The ARDS pod, shown in Fig. 1, configured for installationon high dynamic aircraft weapon wing stations, consists of a multi-channel GPS receiver, Inertial Measurement Unit(IMU), integrated navigation processor, data-link transceiver, data recorder, encryption device and power supplies.It has an independent nose-mounted GPS dual-band antenna, and relies on the aircraft only for prime power. Thesebuilding blocks of the high dynamic TSPI system are also available in various IM configurations for tracking land,sea and airborne vehicles under low, medium and high dynamic environments.

Figure 1. The ARDS TSPI Pod

Page 3: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics3

TSPI Capabilities of ARDS: Initial design of the ARDS GPS receiver was based on five hardware trackingchannels, which was the state-of-the art in the 1980s. During the 1990s, it was replaced by a 10 parallel channeltracking device, made feasible by the advances in GPS technology following the GPS constellation FOC landmark.The GPS receiver is tightly coupled with an IMU thru a 17-State Kalman Filter, which was again a state-of-the-artintegration methodology for the optimum complementary integration of GPS and IMUs, This integration enabledthe pod to provide continuous and smooth TSPI outputs even while enduring dynamic maneuvers. The accuracy ofthe TSPI itself was improved considerably over what can be achieved using absolute mode of GPS, by theimplementation of real-time, designated Method-I, Differential corrections. Two other modes of Differential GPS(DGPS) were implemented for two other groups of users: (1) Those that required accuracy above the level achievedwith real-time DGPS (this was called Method-III), and (2) Those that did not have the need for real-time accuracy atthe participant and can live with differential corrections in the position solution domain at the ground station (thiswas called Method-IV). The details of implementation of these three DGPS methods, as well as the test results of theRAJPO ARDS system have been published in Ref. 1. This reference also describes the intensive and expensiveflight test program during the verification/validation phase, as well as the complex dynamic flight profiles executedby the test aircraft in order to provide confidence in the TSPI accuracy performance. The overall average RMSaccuracy performance demonstrated for all modes of GPS (absolute, differential), can be summarized as in the 2-4 mrange for position, and up to 0.5 m/s for velocity.

Subsystems and Other Technical Capabilities of ARDS: The ARDS system included fully capable referencereceivers for generating differential corrections for real-time as well as for post-mission processing. The data-linksystem (DLS) used an L-band transceivers selectable between 1350 and 1450 MHz. The DLS network used twofrequencies per network that occupied a 6.8 MHz signal bandwidth (measured at -60 dB)., and were required to beseparated by 11 MHz. Spectrum limitations imposed on the users have limited the majority of the frequencies toonly the 1350 to 1390 MHz band. The ARDS DLS line-of-sight (LOS) range is 70 nmi, and is extendible to 350nmi when utilizing 5 relays. The DLS data rate and maximum number of participants it can track is variable: forexample, it can support up to 250 participants at one message per second, or 25 participants at 10 messages persecond. It uses a 250 time slots Time Division Multiple Access (TDMA) signaling format, and can automaticallyallocate the time slots depending upon the mission and participant configurations. The DLS ground system iscomposed of remotes and master ground station, with pre-defined uplink and downlink message formats as welluser-defined downlink formats. For encrypted data requirements the Range Encryption Module (REM) wasdeveloped. The REM is embedded into the DLS ground subsystems as well as the participant packages, therebyproviding encrypted uplink and downlink messages including the over-the-air-transfer (OTAT) of keys for remotekeying and zeroization.

III. CRIIS Technical Capabilities and Requirements for Future T&E

Most of the US Army, Navy, and Air Force T&E ranges are currently using the aging ARDS TSPI system. Astechnology and GPS signal processing inventions advanced, some periodic upgrades have been made to the ARDS.However, the equipment and systems themselves that the T&E ranges are required to test have continued to becomemore accurate, thereby making the “Truth Source” itself obsolete. Under the auspices of the CTEIP JointImprovement Modernization program, numerous T&E community requirements meetings were conducted to clarifyand document the requirements of the future TSPI system, which will replace ARDS. The CRIIS Program Office,replacing the RAJPO, has now initiated the CRIIS program with all the emerging T&E requirements captured intothe Test Capabilities Requirement Document (TCRD) for the CRIIS2. The following sections will describe the areasof enhancements of this TSPI standard for the future, along with the technical challenges that have to be surmountedin order to produce a system with capabilities and functionalities significantly beyond those of the ARDS.

CRIIS Major Areas of Enhancements: The lessons learned from more than a decade of constant use of theARDS standard has been used by the T&E personnel and CRIIS Program to define the scope of CRIIS to enhancefive areas of technology and systems engineering associated with a TSPI truth source:

1. Increased TSPI Accuracy and Faster Data Update Rates: The CRIIS program will develop anumber of TSPI components with differing capabilities/sizes/cost. The more capable componentsare expected to provide real-time position, velocity and attitude an order of magnitude better than

Page 4: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics4

ARDS. Their post-mission accuracy will be at least three times better than their real-time capability.The less capable components will be developed for those users requiring less accuracy, and lowercost. Existing Commercial-Off-The-Shelf (COTS) technology will be used where possible.

2. Enhanced Data Link Capabilities: The CRIIS data link will be characterized by enhanced datatransfer flexibility, control, data capacity and spectral efficiency. The more capable CRIIScomponents will significantly improve the ARDS throughput capability and allow flexibility to varynumber of participants, message lengths, formats and data rates. The detailed requirements arethoroughly discussed in Ref 3.

3. Components Miniaturization and Modularity: This is a key requirement of CRIIS to facilitatemounting of the system inside various low observable (LO) and unmanned vehicles. In addition,existing miniaturized COTS components will be used to instrument dismounted soldiers and spaceconstrained low dynamic vehicles. The components will be procured in a modular form to facilitateplug and play and optimization of assets.

4. Open Architecture Design and Standardization of Interface Protocol: CRIIS will take abuilding block approach to the design of the system. A common framework will be developed thatemploys standard data protocols, interface specifications, modular design techniques etc. Theinterchangeability/interoperability of the modules is very desirable for rapidly tailoring the systemto meet the various needs of the ranges.

5. Enhanced Data Encryption Capability: The technology of the currently used REM for encryptionin the ARDS is obsolete. This technology will be upgraded with new encryption hardware andsoftware resulting in the development and/or integration of a Next Generation Encryption Device(NGED). The upgrade is critically needed to support the higher throughput data link, provideencryption up to the Top Secret level and contribute to the overall standardization of interfaces.

CRIIS TSPI Accuracy Levels and Associated TRLs: The T&E ranges have wide variety of accuracy and datarequirements depending on the platform under test and the objectives of each test mission. These needs vary fromtracking dismounted soldiers, to ground vehicles and ships, to advanced high dynamics aircraft. Testing the latesthigh performance aircraft and weapons systems require very accurate TSPI and very capable data link. The threelevels of TSPI accuracies and parameters required by the various configurations are summarized in Table 1. The

specification of the accuracies at these unprecedented levels is driven by the lessons learned from the ARDS system

Table 1. CRIIS TSPI Accuracy Requirements

Page 5: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics5

development and applications, and the foresight to ‘stay ahead of the technology curve’ of the latest and futureaircraft and weapons. The actual entries in Table 1 are specified using different colors to signify challenges and thelevel of effort/resources that may be required to realize them. The entries in ‘black’ have been reported and havebeen found to be available in a number of fielded systems, although the form and fit may not be consistent withCRIIS requirements. The entries in ‘green’ imply the fact these levels of accuracies for these parameters are beingachieved in many systems, some of which are already integrated while others are in various phases ofdevelopment/simulation/demonstration5,6. The additional development effort required to get them to the productionreadiness level for CRIIS is seen as low risk. The entries in ‘red’ are considered to be not mature enough for CRIISrequirements, and are expected to present moderate to high risk. The ‘red’ entries marked with superscript ‘2’ are invarious levels of research and evaluation under medium dynamics, and are being evaluated using advanced datareduction procedures in the post-mission mode due to non-availability of an integrated suite of suitablecomponents7. The ‘red’ entries marked with 3 and 4 are expected to present some very difficult challenges sincethey are two orders of magnitude more stringent than the most accurate existing systems. Studies and simulationsrequired to mature algorithms to achieve these accuracies are on-going. Efforts to model all the error sources andphenomena (IMU/GPS-antenna lever arm flexures, timing, multipath , high dynamics and signal blockages) areactively being pursued8-9. Finally, for some of these parameters, such as acceleration and attitude rates, it is notknown how they will be validated. The validation of these unprecedented accuracies for a CRIIS system will itselfrequire an even more accurate truth standard! This issue is analyzed in the remainder of this paper along with anefficient and repeatable test methodology.

The CRIIS program management realized the need to asses the risk and feasibility of achieving theseperformance levels using the guidelines provided in the DoD 5000.2-R document2. This document assigns aTechnology Readiness Level (TRL) in the range of 1 to 9 to the various phases of system development leading toproduction, beginning with the very basic technology research phase. The generic technical activities associatedwith these 9 levels are summarized in Figure 3, and were adapted to the specific CRIIS expected technical system

Figure 3. DoD 5000.2-R TRL Definitions

Page 6: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics6

design/development phases. The purpose of using this guideline was to review the state-of-the-art of GPS and INStechnologies which are being pursued by various major companies, small businesses, and academia and establish thereasonableness of the CRIIS documented performance requirements. The information for conducting this assessmentwas readily available from market research, internet search, conference proceedings and technical conferenceexhibitions. The CRIIS program office goal necessitates that all technologies required for implementation of thesystem are at the 6-7 TRL level in order to reduce the cost/schedule risk and deliver a useful suite of equipment.

Review of GPS Signal Modernization and Advanced Signal Processing: The GPS constellation has continuedto improve in signal performance and accuracy, while at the same time a number of advanced data processingalgorithms have been invented and perfected. These developments will make some of the seemingly tough accuracyspecifications for CRIIS a reality with low risk. With new blocks of GPS satellites and Legacy AccuracyImprovement Initiatives (L-AII), the signal-in-space (SIS) error also known as User Range Error (URE) associatedwith the satellite ephemeris/clock data, has continued to decrease as shown in Figure 4. The details of the L-AIIprogram using the existing five GPS monitor stations and additional 14 NGA ground monitor stations along withGPS Control Segment Kalman Filter improvements are available in Ref 10. It can be concluded from this that insome cases it may not be necessary to use DGPS anymore. While in some cases at the other end of the spectrumrequiring extreme accuracies, Real-Time Kinematic (RTK) GPS and the Epoch-By-Epoch (EBE) algorithmsrequiring extensive DGPS reference receiver network are available6. The most advanced measurement processinginventions to emerge in the last few years can be generally grouped into the Ultra-Tightly-Coupled (UTC) method ofGPS/INS integration. Numerous organizations have invented/developed UTC algorithms that vary in the basicformulation of the problem as well as implementation details. The inventors of the algorithms include: AerospaceCorporation, Boeing Co., Draper Labs, Honeywell Corp., L3-IEC, Raytheon Co., and TISI Inc. A review of thepatents from these companies reveals that although the goal of UTC algorithm is the same (increased accuracy, anti-jam etc) the implementation details are unique. The chances of improving TSPI accuracy by a large factor neverlooked better. However, a significant effort will be required to raise the TRL of these technologies to the 6-7 rangefor the CRIIS Level-II/Level-III requirements8,9.

Figure 4. History of GPS Constellation SIS (URE) Performance

Page 7: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics7

The legacy GPS constellation composed of all the blocks prior to the block-IIRM, have provided only threecodes for receiver implementation. All legacy GPS signals included the 50 Hz data, which is a key limitation to thebest achievable accuracy. The block-IIRM (three satellites already so far) provide additional medium/long lengthcivilian codes on the L2 frequency, which currently transmits only the Y-Code. The block-IIF satellites expected tobegin launching in 2008/2009, will broadcast the L5 (1176 MHz) frequency with a dataless channel, that cansignificantly enhance the capabilities of the RTK and EBE techniques. Although these new signals/codes are notoperational yet, the status should be carefully monitored for exploitation by CRIIS.

IV. Why the 10X Accuracy Requirement for a TSPI Standard?It has been historically assumed that a TSPI truth source used for verifying/validating the accuracy of a

navigation/guidance system at the T&E ranges should posses at least ten times the accuracy of the system beingtested. The mathematically rigorous basis for this rule-of-thumb has not been found. Nevertheless, it has beeneffectively used to find and/or devise the proper TSPI source, thus far. With the rapidly advancing performancecapabilities of the weapons and aircraft systems being developed, it will not be possible to continue to find/developsuch a truth source. The dilemma then is, what to use to test these systems? Following the development of the CRIISaccuracy specification shown in Table 1, an analysis of some of the candidate systems to be evaluated at the T&Eranges was conducted to answer this question. An ‘Advantage Factor’ is defined as the ratio of the Root MeanSquared (RMS) values of the SUT and TSPI parameter accuracies, for the purpose of comparing the system undertest (SUT) accuracy to the TSPI standard accuracy:

N = RMS(SUT Parameter)/ RMS(TSPI Parameter) (1)

In order to develop a practical method of data analysis for acceptance of a system against a truth source, a set ofgeneric CRIIS user systems are created with accuracy requirements shown in Table 2. This table also shows theadvantage factors that CRIIS would provide in Level-II and Level-III (the user systems are hypothetical and do notrepresent any actual system). However, the following analysis and conclusions are valid and will be employedduring the CRIIS equipment procurement, and recommended while using CRIIS on the T&E ranges.

Table 2. CRIIS Sample Advantage Factors for Generic User Systems

Page 8: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics8

A comparison of the advantage factors against the 10X rule, shows that for some parameters the CRIIS systemspecification is over specified and for others it is under specified. It is also interesting to note that some of the overspecified parameters can be achieved with the technological advances discussed in section III,.. On the other hand itappears that the risk for achieving some of the under specified parameters may be high even with all the technicalbreakthroughs in GPS. The following examples demonstrate how the modularity of CRIIS and the analysismethodology can be effectively used to meet a variety of T&E user requirements:

• Example 1: For user-1 the CRIIS advantage factor for position is only 3 with both Level-II andLevel-III. While for velocity using Level-II it is 10. However, for this user-1, advantage factor forattitude with Level-II CRIIS is only 2. It will be seen from the analysis developed below that Level-II may be adequate for this user. From the table it can be seen that CRIIS level-III providesadvantage factors of 50 and 100 for the velocity and attitude parameters., which may appear to beextremely desirable but will be quite expensive.

• Example 2: CRIIS advantage factors for position for user-2 are much larger than the 10X standard,Level-II CRIIS provides advantage factor 3 for velocity, but is not adequate for attitude requirement.But Level-III provides advantage factors 15 and 5 for velocity and attitude respectively. It mayappear that a fully capable Level-III may have to be used. However, since the position accuracyrequirements are not very stringent, even conventional DGPS may not be required (see Figure 4).

Statistical Acceptance Criteria for the ‘Truth Source’: The term ‘truth source’ is used in this section to implythe CRIIS system that will be used to verify the system under test, or the system that is required as the truth sourcefor verifying the accuracy of the CRIIS system itself. Hence, the analysis below is applicable to both the situations.In other words, the term ‘System Under Test (SUT)’ may refer to either CRIIS or the system that CRIIS will test.Similarly, the term ‘TSPI’ may refer to the truth source for testing CRIIS or CRIIS itself.

Let XSUT(n) = SUT measured parameter value

XTSPI(n) = TSPI measured parameter value, and

Test Statistic = TS = RMS of (XSUT – XTSPI) (2)

(The above statistical computation is commonly carried out over a large number of samples)

Then, TS2 = E(XSUT - XTSPI )2

= σ 2SUT + m2

SUT +σ 2TSPI + m2

TSPI - 2mSUT mTSPI (3)

Where σ = standard deviation of the SUT or TSPI, and

m = mean of the SUT or TSPI

From equation (1), RMS(SUT Parameter) = N*RMS(TSPI Parameter) (4)

where, N = the advantage factor (which according to the rule-of-thumb is desired to be 10X)

Then equation (2) becomes TS2 = (1+ 1/N2 )(σ 2SUT + m2

SUT) - 2mSUT mTSPI (5)

In practical applications the second term in equation (5) is composed of the mean values of the SUT and the TSPI issmall compared to the first term, and can be ignored as a first approximation. To ensure that this in fact is the case,additional data analyses procedures were carried out, and bias error sources (known as well as unknown)

contributing toa significant mean values are usually determined and accounted for in the analysis.

i.e. TS2 ~ (1+ 1/N2 )(σ 2SUT + m2

SUT) (6)

Page 9: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics9

The factor N can be calculated prior to data reduction by knowing the performance quality of the TSPI and thespecified accuracy of the SUT using equation (1). Note that the quantity TS on the left hand side of equation (6), iscalculated from the data collected during the test, while the quantity on the right hand side of equation (6) iscomputed from specified accuracy parameters of the SUT. It is thus clear that if equation (6) is true using the testdata, then the SUT can be accepted as having satisfied the specified requirements. This relationship is plotted inFigure 5 to gain further insight.

Two significant observations can be made regarding the general relationship between the TS, the SUT and the TSPIvalues based on this Figure 5.

• The TS truly represents the performance of the SUT alone, if the TSPI was perfect, i.e. infinitelyaccurate.

• The RMS of TS approaches the True RMS of the SUT, as N becomes large. At N=10, the error inthe TS representing the true RMS of the SUT is less than 0.5%. Intuitively it appears that this mayhave been the reason for the rule of thumb 10X requirement.

Some significant and practically useful conclusions that can be summarized based on the above equations are asfollows:

• The TSPI source need not be 10X as good as the system under test. Calculating TS from test data andthe TSPI proven specifications, the performance of the SUT can be backed out using equations (4) and(5). This can be compared against the SUT specifications for acceptance.

• In accepting CRIIS, some parameters can be verified using ‘other truth systems’ that possess 10X orhigher advantage factors. For other parameters, e.g. velocity/attitude accuracies in Level-II/Level-III,performance of available truth sources are comparable to those of CRIIS. The above methodology willprovide the correct tools for acceptance.

• There are some performance parameters of CRIIS Level-II/Level-III, e.g. acceleration/attitude rate etc,for which a higher capability truth source is not available. In these cases, two similar CRIIS systems

Figure 5. TS Bound vs TSPI Advantage Factor

Page 10: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics10

may be used to test each other. If the system is performing as designed, equation (5) will yield a valuetwice as large as the CRIIS specification.

For the purpose of illustration, a sample plot of a general measured test parameter is shown in Figure 6 along withthe TSPI and the ‘Absolute Truth’. The standard deviation of the test parameter as shown is 3 times that of the TSPI.Usually the statistics must be computed over a reasonably large number of samples. A smaller set of points canreduce the confidence due to data noise, whereas too large a sample size cannot capture the dynamic properties ofthe participant test trajectory. As part of the CRIIS acceptance, these issues will be carefully addressed, since thereare a number of ways in which the test statistic may be computed. Some possible computations to be used may be asfollows:

• A single RMS statistic for the entire SUT trajectory• A number of RMS statistics computed over consecutive segments of the trajectory characterized by

some dynamic signature (straight and level, turn etc). The segments may be disjoint or overlapped(running window)

• RMS statistics for each point of the trajectory, over a large number of repeatable trajectories. This typeof data reduction is most attractive, and provides a truly robust method of acceptance. However, if datacollection requires actual flights, then care must be taken to make the trajectories as similar aspossible. Even so, the cost of executing such a program will be prohibitive. The concept provided insection V makes this possible.

V HIL Test Concept for CRIIS TMP

As mentioned earlier the CRIIS acceptance testing is required to reduce the risk to production and fielding.Extensive and thorough testing of the implemented algorithms and hardware is expected during the phases ofdevelopment (partially or completely integrated). An exclusively open air flight test program can not only be costprohibitive, but may require the hardware to be integrated and flight qualified. Due to the technical risks andreadiness of the technology already discussed, this approach is not feasible. GPS simulators with extremely highcapabilities are readily available11,12 and are being widely used with advanced external enhancements13 to properly

Figure 6. Example Parameter Measurement Profiles for SUT & TSPI

Page 11: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics11

stress and thoroughly verify the performance specifications of GPS/INS based systems. A Hardware-In-the-Looptest concept, shown in Figures 7 and 8 is developed to effectively and efficiently test the candidate systems.

Figure 7. Proposed Hardware-in-Loop Concept for CRIIS Tests

Figure 8. Notional Differential Network Simulation Concept for CRIIS Tests

Page 12: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics12

Figure 7 shows the overall HIL concept divided into two segments. The segment consisting of the simulator RFsignal and the parameters that generate the signal is labeled ‘Common’. There are a number of factors in the satellitevehicle (SV) constellation simulation that, if not identical for candidate systems, can skew the results. Factors suchas day of test (almanac), satellite power and antenna patterns, ephemeris and clock errors, and atmospheric errors areimportant. Similarly, user trajectory dynamics, flight profiles, GPS antenna location and pattern etc are other criticalparameters that are required to be common. By providing common guidelines and data description for this segmentthe testing phase can ensure an ‘apples-to-apples’ comparison between the competing systems. The other segment islabeled ‘Application/System Unique’, and can consist of components and algorithms developed by the individualCRIIS configurations or candidate level-II/level-III systems. Depending upon the detailed design, a differential GPSreceiver network may be required. The details such as capabilities, number, deployment criteria, data content andrate and storage requirements of the network may vary widely between applications/configurations. So, the nextcritical consideration for a HIL based concept is to develop a simulation of this infrastructure. Such a concept isshown in Figure 8, and exploits the same simulator used to stress the participant receiver of Figure 7. Theconstellation parameters of the simulation will remain exactly the same as before, with the network receiverparameters determined for the configuration/system according to its unique implementation scheme. The networkdata can be recorded and run through the ‘lab integrated’ participant package non real-time. The key requirementhere is to synchronize the two parts of the simulation to a common time-tag, so that in effect the end result will beequivalent to stressing all the components in a consistent manner.

The advantage of HIL based testing for CRIIS is that all the GPS components and the operational processingsoftware can be completely and thoroughly stressed. The IMU is the only item that has to be modeled. This willrequire some limited flight testing. The actual flight profile data can, however, be used to re-run the HIL to verifythe agreement between actual and simulated performance. This approach is very valuable in identifying any mis-modeling and/or algorithmic ‘tuning’ prior to acceptance. The HIL also appears to be a valuable asset for the T&Eranges for future verification, re-calibration of CRIIS, running excursions etc. prior to use in validating othersystems.

VI. Conclusion

The CRIIS program office has developed new consolidated technical requirements for the development of thenext generation TSPI standard, CRIIS. Existing and evolving advanced technologies will be exploited to maximizethe capabilities. The five major areas of enhancement beyond the ARDS system were presented. The stringent TSPIaccuracy requirements have been discussed, and examined with respect to the state-of-the-art of GPS technologyand processing advancements. This paper has concluded that the 10X accuracy requirement for the truth standard isnot imperative anymore, and derived statistical relationships for testing systems against truth standards of equal orbetter performance specifications. HIL test concepts and test methodologies have been presented to thoroughly,effectively and efficiently test the rapidly advancing GPS/INS TSPI systems.

VII. Disclaimer

The statistical analysis and HIL test concepts for CRIIS were developed by the authors as part of their taskrequirements to support the CRIIS program office under the TEAS IV/V contracts. These ideas should not beconstrued as ‘the official CRIIS Program Office position’ for executing the CRIIS Program itself.

References

1Mahmood, S., and Burkett, S., “Range Application Joint Program Office: GPS-Based equipment – A new Proven TSPIBaseline for DoD Test and Training Ranges,” ITEA Journal of Test and Evaluation, 16:2, June/July 1995, pp. 26-31

2Test Capabilities Requirements Document (TCRD) for the Common Range Integrated Instrumentation System (CRIIS),Eglin AFB, FL., 21 September 2006,

3Lunsford, R.L., Bratton, L., “Challenges for Multiple Data Links to Meet The Future Wireless Data Transfer Needs of theTest Range Community,” AIAA US Air Force T&E Days, Destin, FL. Feb. 2007

Page 13: [American Institute of Aeronautics and Astronautics 2007 U.S. Air Force T&E Days - Destin, Florida ()] 2007 U.S. Air Force T&E Days - Common Range Integrated Instrumentation System

American Institute of Aeronautics and Astronautics13

4DoD 5000.2-R, Mandatory Procedures for MDAPS and MAIS Acquisition Programs, Appendix 6, April 5, 2002, pp. 183-185

5Kennedy, S., Hamilton, J., Martell, H.., “Architecture and System Performance of SPAN – NovAtel’s GPS/INS Solution,”

6Bock, Y., Macdonald, T.J., Merts, J.H., Bock, L. “Epoch-By-Epoch Real-Time GPS Positioning in High Dynamics and AtExtended Ranges”, ITEA Journal of Test and Evaluation, 25:3, Sept/Oct 2004, pp37-45

7 GLITE Configuration 2B Concept Test Report, Range Division, Air Force Flight Test center, Edwards AFB, CA., 17 Oct.2005

8Sennott, J., Senffner, D., “A GPS Carrier Phase Processor for Real-Time High Dynamics Tracking,” Proc. 53rd AnnualMeeting of ION, Albuquerque, NM, June/July 1997, pp. 299-308

9Quinn, P., Lewis, D., Berarducci, M. et. al., “Theory & Results of ANTARES Lever-Arm Flexure Estimation for UTCGPS/INS Emitter Geo-Location Systems Using Auxiliary Antenna-Mounted IMUs”, ION GNSS 2006, Ft. Worth, TX., Sept 26-29 2006, pp. 157-167

10Creel, T., “New Improved GPS – The Legacy Accuracy Improvement Initiative,” GPS World Magazine, March 2006,www.gpsworld.com

11SCS 3500 Constellation Simulator Technical Information, L3-IEC Corp., Anaheim, CA., www.iechome.com

12Spirent GSSXXXX Series of Simulators, Spirent Federal Systems Inc., Yorba Linda, CA., Internet address:www.spirentfederal.com

13 Catanzarite, J.,Green, G., Jacobs, D.,“Vertical Sensing Effectiveness and CONOPs Tool for Operational Requirements:VECTOR”, ION-GNSS 2006, Ft. Worth, Tx., Sept 26-29, 2006, pp. 152-156