aiaa-2001-0170 a single-vector force calibration method ...mln/ltrs-pdfs/nasa-aiaa-2001-0170.pdf ·...

AIAA-2001-0170A Single-Vector Force Calibration MethodFeaturing the Modern Design of Experiments(Invited)

P.A. Parker, M. Morton, N. Draper, W. LineNASA Langley Research CenterHampton, Virginia

For permission to copy or republish, contact the American Institute of Aeronautics and Astronautics1801 Alexander Bell Drive, Suite 500, Reston, VA 20191-4344

39th AIAA Aerospace Sciences Meeting & Exhibit8-11 January 2001

Reno, Nevada

1American Institute of Aeronautics and Astronautics

AIAA-2001-0170

A SINGLE-VECTOR FORCE CALIBRATION METHOD FEATURINGTHE MODERN DESIGN OF EXPERIMENTS

P.A. Parker*, M. Morton†, N. Draper‡, W. Line§

NASA Langley Research CenterHampton, Virginia 23681

Abstract

This paper proposes a new concept in force balancecalibration. An overview of the state-of-the-art in forcebalance calibration is provided with emphasis on boththe load application system and the experimental designphilosophy. Limitations of current systems are detailedin the areas of data quality and productivity. A uniquecalibration loading system integrated with formalexperimental design techniques has been developed anddesignated as the Single-Vector Balance CalibrationSystem (SVS). This new concept addresses thelimitations of current systems. The development of aquadratic and cubic calibration design is presented.Results from experimental testing are compared andcontrasted with conventional calibration systems.Analyses of data are provided that demonstrate thefeasibility of this concept and provide new insights intobalance calibration.

Introduction

Direct force and moment measurement of aerodynamicloads is fundamental to wind tunnel testing at NASALangley Research Center (LaRC). An instrumentknown as a force balance provides these measurements.Force balances provide high-precision measurement offorces and moments in six degrees of freedom. Thebalance is mounted internally in the scaled wind tunnelmodel, and the components measured by the forcebalance consist of normal force (NF), axial force (AF),pitching moment (PM), rolling moment (RM), yawingmoment (YM), and side force (SF), see Figure 1.

Figure 1. Measured aerodynamic forces and moments.

Force Balance Background

The force balance is a complex structural springelement designed to deflect a specified amount underapplied load. This deflection of the balance under loadresults in a change in strain within the flexuralelements. Differential strain is measured with sixWheatstone bridges, each consisting of four foilresistive strain gages. Each bridge is designed toprimarily respond to the application of one of the sixcomponents of load. The electrical response of thesebridges is proportional to the applied load on thebalance. This electrically measured strain, as a functionof load, forms the basic concept of force balancemeasurements that has been generally used since the1940s.

The balance flexural elements are optimized such thatthe magnitude of the strain response is approximatelythe same for the individual application of eachcomponent of load. The magnitude of the applied loadin each component is not equivalent, and therefore theflexural elements do not have the same deflection in allaxes. Typical balance load ratios of lift to drag aresixteen to one. These structural requirements dictatethat the force balance must be carefully designed in

* Research Scientist, Model Systems Branch, Member AIAA† Research Statistician, R.J. Reynolds‡ Emeritus Professor, University of Wisconsin, Madison, WI§ President, DOES Institute

Copyright © 2001 by the American Institute of Aeronautics andAstronautics, Inc. No copyright is asserted in the United States underTitle 17, U. S. Code. The U. S. Government has a royalty-free licenseto exercise all rights under the copyright claimed herein forGovernmental Purposes. All other rights are reserved by thecopyright owner.

Balance


order to achieve an accurate measurement of axial force(drag) in the presence of a large normal force (lift).These high ratios of component load also createundesired cross-effects in the balance responses.

Ideally, each balance response signal would respond toits respective component of load, and it would have noresponse to other components of load. This is notentirely possible. The response of a particular balancechannel to the application of other components of loadis referred to as an interaction effect. Balance designsare optimized to minimize these interaction effects.Ultimately, a mathematical model must correct forthese unwanted interaction effects. A calibrationexperiment provides the data to derive an adequatemathematical model.

Force balances are the state-of-the-art instrument usedfor high-precision aerodynamic force measurements inwind tunnel testing. Over the past 60 years, there havebeen many improvements in the areas of balancestructural design and analysis, fabrication techniques,strain sensor technology, data acquisition systems andautomated calibration loading systems. Relatively littlehas changed in the area of force balance calibrationmethodology. The majority of developmental work inthe area of calibration has been focused on automationof load application in order to minimize the calibrationduration, and thereby provide an increase in calibrationfrequency. Regardless of the calibration loadingmechanism, automated or manual, the experimentalapproach to the balance calibration has been the sameand is detailed in the following section.

Balance Calibration Methodology

The balance calibration is the most critical phase in theoverall production of a high quality force transducer.The goal of calibration is to derive a mathematicalmodel that is used during wind tunnel testing of scaledaircraft models to estimate the aerodynamic loading.Furthermore, the accuracy of this model is alsodetermined during the calibration experiment.

In general, force balance calibration consists of settingthe independent variables and measuring the responseof the dependent variables. The applied loads are theindependent variables, and the electrical responses ofthe balance are the dependent variables. Currentmathematical models are based on a polynomialequation where the balance response is a function of theindependent variables.

For k = 2 design variables, a general polynomial can beexpressed by1:

f x

x x

x x

x x

x x x x x x

x

etc

( , )

...

.,

β β

β β

β

β β

β β β β

β

= −

+ +

+ +

+ + + +

+ +

+

0

1 1 2 2

12 1 2

11 1

2

22 2

2

111 1

3

222 2

3

112 1

2

2 122 1 2

2

1111 1

4

(y intercept)

(linear terms)

+ (interaction terms)

(pure quadratic terms)

(cubic terms)

(quartic terms)

where k is the number of independent variables, xi is thei th independent variable, and β represents thecoefficients in the mathematical model. The subscriptnotation of the coefficients is chosen so that each βcoefficient can be easily identified with itscorresponding x term(s). This model can be thought ofas a Taylor’s series approximation to a generalfunction. Typically, the higher the degree of theapproximating polynomial, the more closely the Taylorseries expansion will approximate the truemathematical function. In the current LaRC calibrationapproach, a degree of two is used, and therefore asecond-order model is generated. For balancecalibration, k is equal to six, and the quadraticmathematical model for each response contains a totalof 28 coefficients that consist of the intercept, 6 linearterms, 15 two-way interactions, and 6 pure quadraticterms.

The magnitude of loads set in the calibrationexperiment defines the inference space. The error inthe prediction of the mathematical model is evaluatedwithin this space. The inference space for LaRCcalibrations is based on the plus and minus full-scaledesign loads of the balance. Current force balancecalibration schedules increment one independentvariable at a time. Each independent variable isincremented throughout its full-scale range. During thisincrementing of the primary variable, all other variablesare zero, or are held at a constant magnitude. Thisapproach is referred to as one-factor-at-a-time (OFAT)experimentation.

All of the required set points of independent variablesare grouped into a load schedule, or calibration design.Ordering of the points within the design is based on theefficiency of the load application system and the dataanalysis algorithm. For optimum design execution,efficiency is gained by the application of one load at a


time in incremental levels. Data analysis algorithmsused to determine the balance calibration model oftenrely on a specific grouping of variable combinations.Single variable loading is used to calculate the mainlinear and quadratic effects. Constant auxiliary loads incombination with single incremented loads areperformed in order to assess two-way interactions.

Calibration Systems

In order to set the independent variables of appliedload, a system of hardware is required. The NASALaRC manual calibration consists of a system of levels,free-hanging precision weights, bell cranks, and othermechanical components. After each loading thebalance is re-leveled prior to taking data to assure thatthe applied loads are orthogonal with the balancecoordinate system. This type of calibration system isreferred to as a repositioning system. There are a totalof 81 load sequences performed to calculate thecalibration mathematical model coefficients. Each loadsequence consists of four ascending increments andfour descending increments providing a total of 729data points. Also, four sequences of a multi-componentproof-loading are performed to assess the quality of themath model.2

Manual dead weight balance calibration stands havebeen in use at LaRC since the 1940’s. Figure 2 is aphotograph of one of these systems. They haveundergone continuous improvement in load application,leveling, and data acquisition. These manual systemsare the "standard" that other balance calibration systemshave been compared against. The simple, accuratemethodologies produce high confidence results. Theconventional method is generally accurate, but theprocess is quite complex and labor-intensive, requiringthree to four weeks to complete each full calibration.To ensure accuracy, gravity-based loading is utilized,however this often causes difficulty when applyingloads in three simultaneous, orthogonal axes. Acomplex system of levers, bell-cranks, cables, andoptical alignment devices must be used, introducingincreased sources of systematic error, and significantlyincreasing the time and labor intensity required tocomplete the calibration.

Figure 2. NASA LaRC manual dead-weight calibrationsystem (during multi-component loading).

In recent years, automated balance calibration systemshave been developed.3,4 In general, all automaticbalance calibration systems are designed to simulate themanual calibration process. Unfortunately, theautomation of this tedious manual process results in acomplex mechanical system. Utilizing these automatedsystems, combined with an abbreviated manualcalibration, can reduce the balance calibration time toapproximately two days; however, these new automatedsystems have significant disadvantages.

The mechanical complexity of an automated systemmakes it quite expensive. As compared to manualsystems, this complexity also tends to deteriorate theload application quality due to the added degrees offreedom that factor into the overall accuracy. Sinceoverall system accuracy is based on the elementalaccuracy of multiple high precision load cells andposition sensors, any complex load introduces multiplesources of systematic error. Furthermore, thecalibration system accuracy is difficult toexperimentally verify. Automatic systems cantheoretically compute their system accuracy5 or theycan infer system accuracy by comparing theircalibrations of test balances with calibrations performedusing a traditional manual loading system.

A study was performed by LaRC to evaluate theseautomated systems.6 One conclusion of this study wasthat a mixture of an abbreviated manual calibration andan automated calibration would be required in standardpractice. The manual calibration, in its simplicity,inherently provides more confidence in the datacollected.

Balance(inside fixture)

weights

bell crankassembly

lever arm


Limitations of Current Force Balance Calibration

There are a number of disadvantages to the currentcalibration methodology and load application systems.In regard to methodology, the OFAT approach has beenwidely accepted because of its inherent simplicity andintuitive appeal to the balance engineer. Unfortunately,this approach provides no mechanism to defend againstsystematic errors, common to all calibration systems.These uncontrolled and undetected systematic errors arecontradictory to the foundational premise of the OFATmethodology, that all other variables are held constant.The inability to defend against these errors, determinetheir magnitude and remove their influence on themodel is a significant limitation. Also, the OFATapproach does not provide genuine replicates within thecalibration, and therefore the repeatability of themeasurement environment cannot be adequatelyseparated from the quality of the math model. Thesedata quality issues have been generally overlooked inthe field of force balance calibration. Formalexperimental design offers techniques to address thesedata quality issues as well as provide the balanceengineer with new insights into force balancecalibration. A development of these concepts asapplied to balance calibration is provided in subsequentsections.

Load application systems, both manual and automated,also have disadvantages. The manual systems,although generally considered accurate, are slow andtedious and provide many opportunities for systematicerror. Automated systems that greatly reducecalibration time include additional sources ofsystematic error due to their mechanical complexity,and their expense makes them prohibitive for widespread use. Both of these hardware systems weredesigned around the OFAT calibration designrequirement to set independent variables one at a timeand to obtain maximum efficiency of data collection.

A new comprehensive view to force balance calibrationwas required to address these limitations. This newapproach should simultaneously address the dataquality and productivity shortcomings of the currentcalibration systems.

Single-Vector Calibration System

An innovative approach to balance calibration has beenproposed. This new calibration system integrates aunique load application system with formalexperimental design methodology. The Single VectorBalance Calibration System (SVS) enables the

complete calibration of a six component force balancewith a single force vector. A primary advantage to thisload application system is that it improves on the"trusted" aspects of current manual calibration system,and therefore provides high confidence in the results.Applying formal experimental design methodology tothe balance calibration experiment has many featuresthat enable higher quality mathematical models. It alsoprovides new insight into calibration data regarding theadequacy of the math model, and the repeatability ofthe measurement environment. Moreover, formalexperiment design techniques provide an objectivemeans for the balance engineer to evaluate and reportthe results of a calibration experiment. The SVSimproves data quality, while simultaneously improvingproductivity and reducing cost. A description of theload application system will now presented followed bya description of the application of formal experimentaldesign methodology.

Single-Vector Load Application System

The objectives of the SVS are to provide a calibrationsystem that enables the efficient execution of a formalexperimental design, be relatively inexpensive tomanufacture, require minimal time to operate, andprovide a high level of accuracy in the setting of theindependent variables. Photographs of the assembledsystem are provided in Figure 3.

The individual hardware components of this system arecritical to setting the independent variables rapidly andwith high accuracy. The primary components include anon-metric positioning system, a multiple degree offreedom load positioning system, a three-axisorthogonal angular measurement system7, andcalibrated weights. These components can be seen inFigure 3. All of the system components including thedata acquisition system are designed to meet or exceedthe current requirements for the LaRC manualcalibration system. The system features significantlyfewer components then the LaRC manual system andtherefore fewer sources of systematic error.

The system allows for single vector calibration,meaning that single, calibrated dead-weight loads areapplied in the gravitational direction generating sixcomponent combinations of load relative to thecoordinate system of the balance. By utilizing thissingle force vector, load application inaccuraciescaused by the conventional requirement to generatemultiple force vectors are fundamentally reduced. Theangular manipulation of the balance, combined with theload point positioning system, allows the uni-


directional load to be used to produce three forcevectors (normal force, axial force, side force) and threemoment vectors (pitching moment, rolling moment,yawing moment), with respect to the balance momentcenter. The non-metric positioning system rotates theforce balance being calibrated about three axes. Therotation angle of the balance coordinate system relativeto the gravitational force vector is measured using thethree-axis angle measurement system. A multipledegree of freedom load positioning system utilizes anovel system of bearings and knife-edge rocker guidesto maintain the load orientation, regardless of theangular orientation of the balance. As a result, the useof a single calibration load reduces the set-up time forthe multi-axis load combinations required to execute aformal experimental design.

(a) Photograph of complete mechanical system.Figure 3. Single-Vector Balance Calibration System.

(b) Close-up of load positioning system.Figure 3. concluded.

The application of all six components of force andmoment on a balance with a single force vector poses aphysics-based constraint that warrants a brief discussionin light of its impact on the development of thecalibration design. Any system of forces and moments,however complex, may be reduced to an equivalentforce-couple system acting at a given point. However,for a system of forces and moments to be reduced to asingle resultant force vector acting at a point, thesystem must be able to be reduced to a resultant forcevector and a resultant moment vector that are mutuallyperpendicular.8 Therefore, an infinite number ofcombinations of the six components can be loaded witha single force vector, but any arbitrary combination offorces and moments cannot. Figure 4 provides theLaRC force balance coordinate system used for thefollowing derivation of the governing equations.

weights

loadpositioningsystem

non-metricpositioning system

balance

balance(inside fixture) angle measurement

system


+NF

+AF

+SF +PM

+YM

+RM

– z

– y

+ x

Balance Moment Center

Figure 4. LaRC force balance coordinate system.

The perpendicular relationship can be expressed as thedot product of the resultant force vector and theresultant moment vector. By, setting the cosine of theangle between the two vectors equal to zero, thefollowing relationship can be derived.

− + −

+ + + +=( )( )

( )( ) ( )( ) ( )( )RM AF PM SF YM NF

RM PM YM AF SF NF2 2 2 2 2 20 (1)

where, NF, AF, PM, RM, YM, SF are the componentloads in actual units. Therefore for non-zeromagnitudes of the resultant force and moment vector,Equation 1 can be simplified as follows.

− + − =( )( ) ( )( ) ( )( )RM AF PM SF YM NF 0 (2)

This governing equation constrains the relativedirection and magnitude of each of the six componentsof load applied to a balance with a single force vector.It also introduces a co-linearity between the threeproducts of Equation 2. The implications of this co-linearity are discussed later in this paper.

Furthermore, if the system of forces and moments canbe reduced to a single force vector, then there are aninfinite number of points in which that resultant forcevector can be loaded to generate the desired moments.The following equation governs the location of thepoint in which the forces can be loaded to generate thedesired moments.

0

0

0

− −

− =

NF SF

NF AF

SF AF

x

y

z

RM

PM

YM

(3)

where, x , y , and z are the linear distances from thebalance moment center.

The three balance force components are a function ofthe applied load and the angular orientation of thebalance in three-dimensional space. To generate adesired combination of the three forces, the balance ismanipulated to a prescribed angular orientation usingthe non-metric positioning system, and the orientationof the balance is precisely measured on the metric endusing the angle measurement system. This anglemeasurement system provides the components of thegravitational vector projected onto the three-axes of thebalance coordinate system. The magnitude of theapplied load is known based on the use of calibrateddead-weights. Combining the measured gravitationalcomponents on the balance axes and the known dead-weight enables the determination of the three forcecomponents.

The three balance moment components are a functionof the three force vectors and the position of the pointof load application in three-dimensional space relativeto the balance moment center. The balance momentcenter is a defined location in the balance coordinatesystem that serves as a reference point in which themoment components are described. The point of loadapplication is set using the multiple degree-of-freedomload positioning system. This system defines the x, y,and z distances from the balance moment center to thepoint of load application. Due to the degrees offreedom in the load positioning system, the point ofload application is independent of the angularorientation of the balance. Stated another way, whenthe balance is manipulated in three-dimensional space,the point of load application remains constant.

The SVS achieves the objectives of rapid and accuratesetting of the independent variables. Even though thisload application system would greatly enhance theexecution of the current OFAT design, it is particularlywell suited to meet the requirements for the executionof a formal experimental design.


Formal Experimental DesignApplied to Balance Calibration

LaRC has been conducting research in a "moderndesign of experiments" (MDOE) approach to forcebalance calibration. Force balance calibration is anexperiment in which independent variables (appliedloads) are set and dependent variables (balanceelectrical responses) are measured. A six dimensionalresponse surface that models the electrical responses asa function of applied load is desired. Formalexperimental design techniques provide an integratedview to the force balance calibration process. Thisscientific approach applies to all three major aspects ofan experiment; the design of the experiment, theexecution of the experiment, and the statistical analysesof the data.

Formal experimental design has been applied to windtunnel testing at NASA LaRC since January 1997.9

The application of formal experiment design to balancecalibration was first considered in 1980 at NASALaRC.10 A classical quadratic experimental design wasconducted in 1980, and the results appeared promising.However, due to the complex load applicationrequirements of an MDOE design, this effort wasconsidered not to be feasible for general application,and the idea remained dormant for nearly two decades.Then, the single-vector calibration method wasconceived to provide an efficient means to execute aformal experimental design. To satisfy therequirements of this unique load application system, acustom MDOE calibration design was developed.

An MDOE approach deviates from the current trend ofcollecting massive data volume, by specifying ampledata to meet requirements quantified in the designwithout prescribing volumes of data far in excess of thisminimum. The goal is to efficiently achieve theprimary objective of the calibration experiment; namelythe determination of an accurate mathematical model toestimate the aerodynamic loads from measured balanceresponses.

Modern Design of Experiments Fundamentals

The three fundamental quality-assurance principles ofMDOE are randomization, blocking, and replication.11

Randomization of point ordering ensures that a givenbalance load is just as likely to be applied early as late.If sample means are stable, the point ordering does notmatter. However, if some systematic variation(instrumentation drift, temperature effects, operatorfatigue, etc) causes earlier measurements to be biased

low and later measurements to be biased high thenrandomization converts such unseen systematic errorsto an additional component of simple random error.Random error is easy to detect and also easy to correctby replication and other means. Randomization of pointordering also increases the statistical independence ofeach data point in the design. This statisticalindependence is often assumed to exist in the currentmethods of balance calibration, but systematic variationcan cause measurement errors to be correlated andtherefore not independent of each other, as required forstandard precision interval computations and othercommon variance estimates to be valid. Even relativelymild correlation can corrupt variance estimatessubstantially, introducing significant errors intoestimates of "95% confidence intervals" and other suchquality metrics.

Blocking entails organizing the design into relativelyshort blocks of time within which the randomization ofpoint ordering ensures stable sample means andstatistical independence of measurements. Whilerandomization defends against systematic within-blockvariation, substantial between-block systematicvariation is also possible. For example, calibrationsspanning days or weeks might involve differentoperators, who each use slightly different techniques, orpossess somewhat different skill levels. By blockingthe design, it is possible in the analysis to remove thesebetween-block components of what would otherwise beunexplained variance.

Replication causes random errors to cancel. Thisincludes otherwise undetectable systematic variationthat is converted to random error by randomizing thepoint order of the loading schedule. Replication alsofacilitates unbiased estimates of what is called "pureerror" - the error component due to ordinary chancevariations in the data. These pure-error estimates arecritical to evaluating the quality of the calibrationmodel by permitting the fit of the model to the data tobe assessed objectively.

Pure-error estimates make it possible to separateunexplained variance into lack-of-fit and pure-errorcomponents. The basis of investigating higher ordermath models is that the lack-of-fit component issignificant in current quadratic models, meaning largecompared to the pure-error component. If the lack-of-fit is significant, then higher order models can be usedto improve balance load estimation quality. If the lack-of-fit is not significant, then adding additional terms tothe math model is not justified.


The volume of data required in an MDOE calibrationdesign depends on three primary parameters: 1) therepeatability of the measurement environment, 2) theprecision requirement, and 3) the inference error risk.The repeatability of the measurement environment is afunction of how repeatable the independent variablescan be set, and the balance responses can be measured.The precision requirement is commonly thought of asthe required balance accuracy, or the desired quality ofthe prediction from the math model. Inference errorrisk refers to the probability that the application of thederived mathematical model used to predict a responsewould produce a result outside of the precisionrequirement. Combining these parameters with theform of the mathematical model and the constraintsinvolved in the execution of the design provide theexperiment designer with the necessary information todefine the data volume required for an adequate design.

Prior to the execution of the design, the quality of thedesign is evaluated. The goal of this evaluation is toverify the adequacy of the design to estimate all of therequired coefficients in the proposed math model. Thisphase does not require experimental data; rather it is anevaluation of the design itself. There are a number ofmeasures that provide insight into the design thatinclude the distribution of unit standard error, and thecomputation of variance inflation factors that are ameasure of co-linearity between the terms in the model.

The analysis of the data obtained from an MDOEdesign is quite different from current balance dataprocessing. This analysis of the data involves statisticaltools, used in tandem, with the experience of thebalance engineer to objectively determine thecalibration coefficients. The number of terms in themodel is minimized by eliminating those withcoefficients that are too small to resolve with asufficiently high level of confidence. Minimizing thenumber of terms in the model lowers the averagevariance, because each coefficient carries someuncertainty.

Also, the total unexplained variance is partitioned intothe pure-error and lack-of-fit components. It isgenerally agreed that we cannot fit the data with amodel better than we can repeat the experimental data.An analysis of unexplained variance provides a methodto make objective judgements about the adequacy of themodel and the potential for improving the model withhigher order terms.

This is merely a sampling of the full "toolbox" ofstatistical analysis and diagnostic techniques that can be

applied to the data obtained from an MDOE design. Itis important to note that the application of thesetechniques is tied directly to the manner in which theexperiment is performed. With such procedures asrandomization and blocking, these techniques producehigh-quality results that defend against the kinds ofsystematic variations that are common in long-durationexperiments such as a balance calibration. Many ofthese analysis techniques are demonstrated later in theanalyses of data section in this paper.

Development of the Single-Vector Calibration Design

The generation of the experimental design for the singlevector system proved challenging due to the uniquephysics-based constraint imposed on the set pointcombinations. An orthogonally blocked third orderexperiment design was required for six independentvariables (NF, AF, PM, RM, YM, SF) that could be usedto determine force balance calibration mathematicalmodels for each response. The experiment designcontains four blocks of thirty-one points each. Itfeatures sufficient degrees of freedom to partition thetotal residual mean square error into pure-error andlack-of-fit error components. All points within thedesign satisfy the physics-based loading constraint.Furthermore, the third order portion of the design canbe performed sequentially after the second orderportion. This allows the second order model to bedetermined and then a decision made about thenecessity of higher order terms.

A literature search was performed to determine thestate-of-the-art in third order designs. There was amodestly large amount of literature describing thirdorder designs. However, the references in the literaturefor six variable designs are a little more modest. Inorder to satisfy the physics-based constraint, to makethe force and moment vectors orthogonal to oneanother, there were no references found.

The approach taken was to generate the design in twodifferent parts, treating the moment and force vectorsseparately. Focussing on the three moment vectorvariables a central composite design was generated inthe moment vector variables. For each point in themoment vector design, there are two dimensions in theforce vector variables that are orthogonal to the momentvector (that is, there is a two-dimensional spaceorthogonal to every vector in three dimensions). Then acentral composite design was generated in the twoorthogonal dimensions for each point in the momentvector variable design. This process was then repeatedin the force vector variables.


Following the procedure described above generates 112points in the quadratic design. For symmetry, the sameprocess is carried out interchanging RM,PM,YM andAF,SF,NF to generate an additional 112 points for atotal of 224 points in the cubic design. It is desirable toexecute the calibration experiment sequentially. Thatis, a quadratic design can be executed first followed bythe remaining points of a cubic design if a lack-of-fittest indicates that a third order model is necessary. Anapproach that can be taken in generating a sequentialdesign is to assign the factorial portion of the design tothe quadratic portion of the model and the axial pointsto the cubic portion. The points in the first four blockscontain the quadratic portion of the design and theremaining four blocks can be added to allow estimationof the cubic terms.

The design above was generated subject to the physics-based constraint in Equation 2. This is a constraint inthe cross-product terms, so only two of the three cross-product terms in the constraint equation can beestimated at one time. One regression degree offreedom is lost due to this constraint in a quadraticmodel. This constraint equation can be multiplied byany of the six variables. For example, multiply it byAF produces the equation below.

− + − =( )( )( ) ( )( )( ) ( )( )( )RM AF AF PM SF AF YM NF AF 0(4)

Only two of the three third order terms can be estimatedat one time. That happens similarly for the other fivevariables, and a total of six of the third order parameterscannot be estimated.

The loss of the single degree of freedom in thequadratic design and six additional degrees of freedomin the cubic design only poses a problem if all of thesecoefficients are significant in the model. In otherwords, for the quadratic model, if at least one of thethree two-way interaction terms in Equation 2 is equalto zero, then all of the significant model coefficientscan be estimated. In the experimental testing to date,this has been the case. For the case of all threeinteractions being significant to the model, ageneralized solution approach was required. Twodifferent approaches have been derived. First, anadditional hardware application sub-system wasdevised that allows for the setting of specificcombinations of independent variables that deviate

from the constraint imposed by Equation 2. This is notconsidered the optimum approach since it deterioratesthe simplicity of the SVS. Alternatively, an iterativeapproach to estimating the model coefficients based onall possible permutations of selecting two of the threecross-product interactions at one time can also beemployed. The model coefficients can be estimatedbased on setting one of the three interactions to zero.Using this approach provides two estimates of each ofthree interactions in the quadratic model. The balanceengineer can then decide which combination of theseinteractions provides the most useful model. Using oneof these two approaches will enable the estimation ofall significant quadratic and cubic interactions withoutthe limitation caused by Equation 2.

The quadratic design for the balance used in theexperimental testing is provided in Table I. The designis listed in standard order, but was executed based on arandomization of the points within each of the fourorthogonal blocks.

A common visualization scheme used in the review ofbalance calibration designs is based on plotting allpossible two variable combinations. For a sixcomponent balance, this requires 15 plots. The LaRCOFAT 729-point design and the MDOE 124-pointdesign are plotted using this technique in Figure 5 and6, respectively. These plots contain the values of theactual set points of the independent variables. It isobvious from these plots that the MDOE 124-point isbetter distributed and more symmetric than the OFAT729-point design.

There are two significant aspects of the MDOE 124-point design that can not be seen using this plottingmethod. First, a comparison of the point orderingcannot be seen. The MDOE 124-point design randomlysamples these variable combinations within a block aspreviously described. The OFAT 729-point designmethodically moves along the series of points on theseplots. Second, the MDOE 124-point is actuallydistributed in six dimensional space which makesvisualization difficult. This six dimensionaldistribution can be seen in Table I, which consists ofmany multiple component combinations in contrast tothe OFAT 729-point design, which consists primarily ofone and two component combinations.


(a) Blocks one and two of the design.Table I. MDOE 124-point quadratic design for Balance NTF-107.

Standard Coded Variables (percent of full-scale load) Actual Units (pounds or inch-pounds)Order Block NF AF PM RM YM SF NF AF PM RM YM SF

1 1 -0.22 -0.87 -0.71 -0.71 -0.71 -0.43 -35 -44 -177 -71 -88 -352 1 0.22 -0.87 -0.71 -0.71 0.71 -0.43 35 -44 -177 -71 88 -353 1 0.33 -0.94 0.71 -0.71 -0.71 -0.09 52 -47 177 -71 -88 -74 1 -0.33 -0.94 0.71 -0.71 0.71 -0.09 -52 -47 177 -71 88 -75 1 -0.33 -0.94 -0.71 0.71 -0.71 -0.09 -52 -47 -177 71 -88 -76 1 0.33 -0.94 -0.71 0.71 0.71 -0.09 52 -47 -177 71 88 -77 1 0.22 -0.87 0.71 0.71 -0.71 -0.43 35 -44 177 71 -88 -358 1 -0.22 -0.87 0.71 0.71 0.71 -0.43 -35 -44 177 71 88 -359 1 -0.45 0.00 0.00 -1.00 0.00 -0.89 -72 0 0 -100 0 -72

10 1 -0.45 0.00 0.00 1.00 0.00 -0.89 -72 0 0 100 0 -7211 1 -0.30 -0.95 -1.00 0.00 0.00 0.00 -48 -48 -250 0 0 012 1 -0.30 -0.95 1.00 0.00 0.00 0.00 -48 -48 250 0 0 013 1 0.00 -0.85 0.00 0.00 -1.00 -0.53 0 -42 0 0 -125 -4214 1 0.00 -0.85 0.00 0.00 1.00 -0.53 0 -42 0 0 125 -4215 1 -0.71 -0.71 -0.38 -0.91 -0.16 -0.71 -113 -35 -96 -91 -20 -5716 1 0.71 -0.71 -0.38 -0.91 0.16 -0.71 113 -35 -96 -91 20 -5717 1 -0.71 -0.71 -0.27 -0.84 0.48 0.71 -113 -35 -67 -84 60 5718 1 0.71 -0.71 -0.27 -0.84 -0.48 0.71 113 -35 -67 -84 -60 5719 1 -0.71 0.71 -0.27 -0.84 -0.48 -0.71 -113 35 -67 -84 -60 -5720 1 0.71 0.71 -0.27 -0.84 0.48 -0.71 113 35 -67 -84 60 -5721 1 -0.71 0.71 -0.38 -0.91 0.16 0.71 -113 35 -96 -91 20 5722 1 0.71 0.71 -0.38 -0.91 -0.16 0.71 113 35 -96 -91 -20 5723 1 0.00 -1.00 -0.45 0.00 -0.89 0.00 0 -50 -112 0 -112 024 1 0.00 1.00 -0.45 0.00 -0.89 0.00 0 50 -112 0 -112 025 1 0.00 0.00 0.00 -0.78 -0.62 -1.00 0 0 0 -78 -78 -8026 1 0.00 0.00 0.00 -0.78 -0.62 1.00 0 0 0 -78 -78 8027 1 -1.00 0.00 -0.37 -0.93 0.00 0.00 -160 0 -93 -93 0 028 1 1.00 0.00 -0.37 -0.93 0.00 0.00 160 0 -93 -93 0 029 1 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 030 1 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 031 1 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 032 2 -0.33 0.94 -0.71 -0.71 -0.71 -0.09 -52 47 -177 -71 -88 -733 2 0.33 0.94 -0.71 -0.71 0.71 -0.09 52 47 -177 -71 88 -734 2 0.22 0.87 0.71 -0.71 -0.71 -0.43 35 44 177 -71 -88 -3535 2 -0.22 0.87 0.71 -0.71 0.71 -0.43 -35 44 177 -71 88 -3536 2 -0.22 0.87 -0.71 0.71 -0.71 -0.43 -35 44 -177 71 -88 -3537 2 0.22 0.87 -0.71 0.71 0.71 -0.43 35 44 -177 71 88 -3538 2 0.33 0.94 0.71 0.71 -0.71 -0.09 52 47 177 71 -88 -739 2 -0.33 0.94 0.71 0.71 0.71 -0.09 -52 47 177 71 88 -740 2 0.45 0.00 0.00 -1.00 0.00 -0.89 72 0 0 -100 0 -7241 2 0.45 0.00 0.00 1.00 0.00 -0.89 72 0 0 100 0 -7242 2 -0.30 0.95 -1.00 0.00 0.00 0.00 -48 48 -250 0 0 043 2 -0.30 0.95 1.00 0.00 0.00 0.00 -48 48 250 0 0 044 2 0.00 0.85 0.00 0.00 -1.00 -0.53 0 42 0 0 -125 -4245 2 0.00 0.85 0.00 0.00 1.00 -0.53 0 42 0 0 125 -4246 2 -0.71 -0.71 -0.27 0.84 -0.48 -0.71 -113 -35 -67 84 -60 -5747 2 0.71 -0.71 -0.27 0.84 0.48 -0.71 113 -35 -67 84 60 -5748 2 -0.71 -0.71 -0.38 0.91 0.16 0.71 -113 -35 -96 91 20 5749 2 0.71 -0.71 -0.38 0.91 -0.16 0.71 113 -35 -96 91 -20 5750 2 -0.71 0.71 -0.38 0.91 -0.16 -0.71 -113 35 -96 91 -20 -5751 2 0.71 0.71 -0.38 0.91 0.16 -0.71 113 35 -96 91 20 -5752 2 -0.71 0.71 -0.27 0.84 0.48 0.71 -113 35 -67 84 60 5753 2 0.71 0.71 -0.27 0.84 -0.48 0.71 113 35 -67 84 -60 5754 2 0.00 -1.00 -0.45 0.00 0.89 0.00 0 -50 -112 0 112 055 2 0.00 1.00 -0.45 0.00 0.89 0.00 0 50 -112 0 112 056 2 0.00 0.00 0.00 0.78 -0.62 -1.00 0 0 0 78 -78 -8057 2 0.00 0.00 0.00 0.78 -0.62 1.00 0 0 0 78 -78 8058 2 -1.00 0.00 -0.37 0.93 0.00 0.00 -160 0 -93 93 0 059 2 1.00 0.00 -0.37 0.93 0.00 0.00 160 0 -93 93 0 060 2 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 061 2 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 062 2 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 0


(b) Blocks three and four of the design.Table I. concluded.

Standard Coded Variables (percent of full-scale load) Actual Units (pounds or inch-pounds)Order Block NF AF PM RM YM SF NF AF PM RM YM SF

63 3 0.33 -0.94 -0.71 -0.71 -0.71 0.09 52 -47 -177 -71 -88 764 3 -0.33 -0.94 -0.71 -0.71 0.71 0.09 -52 -47 -177 -71 88 765 3 -0.22 -0.87 0.71 -0.71 -0.71 0.43 -35 -44 177 -71 -88 3566 3 0.22 -0.87 0.71 -0.71 0.71 0.43 35 -44 177 -71 88 3567 3 0.22 -0.87 -0.71 0.71 -0.71 0.43 35 -44 -177 71 -88 3568 3 -0.22 -0.87 -0.71 0.71 0.71 0.43 -35 -44 -177 71 88 3569 3 -0.33 -0.94 0.71 0.71 -0.71 0.09 -52 -47 177 71 -88 770 3 0.33 -0.94 0.71 0.71 0.71 0.09 52 -47 177 71 88 771 3 -0.45 0.00 0.00 -1.00 0.00 0.89 -72 0 0 -100 0 7272 3 -0.45 0.00 0.00 1.00 0.00 0.89 -72 0 0 100 0 7273 3 0.30 -0.95 -1.00 0.00 0.00 0.00 48 -48 -250 0 0 074 3 0.30 -0.95 1.00 0.00 0.00 0.00 48 -48 250 0 0 075 3 0.00 -0.85 0.00 0.00 -1.00 0.53 0 -42 0 0 -125 4276 3 0.00 -0.85 0.00 0.00 1.00 0.53 0 -42 0 0 125 4277 3 -0.71 -0.71 0.27 -0.84 0.48 -0.71 -113 -35 67 -84 60 -5778 3 0.71 -0.71 0.27 -0.84 -0.48 -0.71 113 -35 67 -84 -60 -5779 3 -0.71 -0.71 0.38 -0.91 -0.16 0.71 -113 -35 96 -91 -20 5780 3 0.71 -0.71 0.38 -0.91 0.16 0.71 113 -35 96 -91 20 5781 3 -0.71 0.71 0.38 -0.91 0.16 -0.71 -113 35 96 -91 20 -5782 3 0.71 0.71 0.38 -0.91 -0.16 -0.71 113 35 96 -91 -20 -5783 3 -0.71 0.71 0.27 -0.84 -0.48 0.71 -113 35 67 -84 -60 5784 3 0.71 0.71 0.27 -0.84 0.48 0.71 113 35 67 -84 60 5785 3 0.00 -1.00 0.45 0.00 -0.89 0.00 0 -50 112 0 -112 086 3 0.00 1.00 0.45 0.00 -0.89 0.00 0 50 112 0 -112 087 3 0.00 0.00 0.00 -0.78 0.62 -1.00 0 0 0 -78 78 -8088 3 0.00 0.00 0.00 -0.78 0.62 1.00 0 0 0 -78 78 8089 3 -1.00 0.00 0.37 -0.93 0.00 0.00 -160 0 93 -93 0 090 3 1.00 0.00 0.37 -0.93 0.00 0.00 160 0 93 -93 0 091 3 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 092 3 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 093 3 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 094 4 0.22 0.87 -0.71 -0.71 -0.71 0.43 35 44 -177 -71 -88 3595 4 -0.22 0.87 -0.71 -0.71 0.71 0.43 -35 44 -177 -71 88 3596 4 -0.33 0.94 0.71 -0.71 -0.71 0.09 -52 47 177 -71 -88 797 4 0.33 0.94 0.71 -0.71 0.71 0.09 52 47 177 -71 88 798 4 0.33 0.94 -0.71 0.71 -0.71 0.09 52 47 -177 71 -88 799 4 -0.33 0.94 -0.71 0.71 0.71 0.09 -52 47 -177 71 88 7

100 4 -0.22 0.87 0.71 0.71 -0.71 0.43 -35 44 177 71 -88 35101 4 0.22 0.87 0.71 0.71 0.71 0.43 35 44 177 71 88 35102 4 0.45 0.00 0.00 -1.00 0.00 0.89 72 0 0 -100 0 72103 4 0.45 0.00 0.00 1.00 0.00 0.89 72 0 0 100 0 72104 4 0.30 0.95 -1.00 0.00 0.00 0.00 48 48 -250 0 0 0105 4 0.30 0.95 1.00 0.00 0.00 0.00 48 48 250 0 0 0106 4 0.00 0.85 0.00 0.00 -1.00 0.53 0 42 0 0 -125 42107 4 0.00 0.85 0.00 0.00 1.00 0.53 0 42 0 0 125 42108 4 -0.71 -0.71 0.38 0.91 0.16 -0.71 -113 -35 96 91 20 -57109 4 0.71 -0.71 0.38 0.91 -0.16 -0.71 113 -35 96 91 -20 -57110 4 -0.71 -0.71 0.27 0.84 -0.48 0.71 -113 -35 67 84 -60 57111 4 0.71 -0.71 0.27 0.84 0.48 0.71 113 -35 67 84 60 57112 4 -0.71 0.71 0.27 0.84 0.48 -0.71 -113 35 67 84 60 -57113 4 0.71 0.71 0.27 0.84 -0.48 -0.71 113 35 67 84 -60 -57114 4 -0.71 0.71 0.38 0.91 -0.16 0.71 -113 35 96 91 -20 57115 4 0.71 0.71 0.38 0.91 0.16 0.71 113 35 96 91 20 57116 4 0.00 -1.00 0.45 0.00 0.89 0.00 0 -50 112 0 112 0117 4 0.00 1.00 0.45 0.00 0.89 0.00 0 50 112 0 112 0118 4 0.00 0.00 0.00 0.78 0.62 -1.00 0 0 0 78 78 -80119 4 0.00 0.00 0.00 0.78 0.62 1.00 0 0 0 78 78 80120 4 -1.00 0.00 0.37 0.93 0.00 0.00 -160 0 93 93 0 0121 4 1.00 0.00 0.37 0.93 0.00 0.00 160 0 93 93 0 0122 4 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 0123 4 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 0124 4 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0 0 0 0


Figure 5. OFAT 729-point design for Balance NTF-107.

Component 1 (percent of full-scale load) versus Component 2 (percent of full-scale load).


Figure 6. MDOE 124-point quadratic design for Balance NTF-107.

Component 1 (percent of full-scale load) versus Component 2 (percent of full-scale load).


A statistical summary of the quadratic design isprovided in Table II.

Table II. Summary of the 124-point quadratic design.Total Degrees of Freedom 123

Regression Degrees of Freedom 26Pure Error Degrees of Freedom 11Lack of Fit Degrees of Freedom 86

Average Leverage 0.217Average Prediction Variance 1.217

Maximum Prediction Variance 1.264

In the event that the loss in regression degrees offreedom results in an inadequate model, as previouslydiscussed, an augmented design was created. Thisaugmented design adds twelve additional points to theoriginal design. In order to set these variablecombinations during the experiment, an additional loadapplication system was devised that allows for theapplication of a co-linear resultant force and momentvector. This load application system utilizes a novelsystem of permanent magnets arranged in a repellingorientation to generate a non-contact force vector that isperpendicular to the gravitational force vector.

A summary of the augmented design for the quadraticmodel is provided in Table III.

Table III. Summary of the 136-point quadraticaugmented design.

Total Degrees of Freedom 135Regression Degrees of Freedom 27Pure Error Degrees of Freedom 11Lack of Fit Degrees of Freedom 97

Average Leverage 0.206Average Prediction Variance 1.206

Maximum Prediction Variance 1.384

Experimental Testing

A sequence of experimental tests was performed tocompare the SVS to the LaRC manual calibrationsystem. The goals of these tests were to verify theperformance of the SVS and to compare themathematical model derived from the MDOE baseddesign to the OFAT design. The preparation of the testbalance and LaRC standard calibration is describedfollowed by a presentation of the experimental testingof the SVS. This is followed by analyses of the datafrom these two methodologies.

Experimental Apparatus

The balance selected for the experimental testing wasthe NTF-107. The NTF-107 is a state-of-the-art sixcomponent balance of monolithic construction. Thefull-scale load range of the balance is provided in TableIV.

Table IV. Test balance full-scale load range.Component Full-scale

design loadNormal Force (pounds) 160Axial Force (pounds) 50

Pitching Moment (inch-pounds) 250Rolling Moment (inch-pounds) 100Yawing Moment (inch-pounds) 125

Side Force (pounds) 80

The balance was refurbished prior to testing to ensurethat its performance would not be questionable duringthe series of experiments performed. This included arefurbishment of the mechanical interfaces and theinstallation of new instrumentation. Balance stabilityand repeatability were considered vital to thecomparison, since the testing took place over an eight-month period. This particular balance was chosen dueto its availability and load range, which werecompatible with the testing schedule and loadapplication system design.

A LaRC manual calibration was performed on the NTF-107 in March 2000. This was a full calibration thatincluded all 729 data points in the OFAT calibrationdesign as well as proof load sequences. The calibrationrequired three weeks to perform. This manualcalibration was considered the baseline for comparisonto the SVS. Figure 2 is a photograph of theexperimental setup for the LaRC manual calibration.

The NTF-107 was then calibrated using the SVS inNovember 2000. Three calibrations were performedand two different calibration designs were evaluated.The results presented in this paper are from the MDOE124-point quadratic design that has been previouslypresented. The points within each of the four blockswere randomized to ensure statistical independence.Each of the four orthogonal blocks requiredapproximately four hours to complete. Prior to theexecution of the design, a tare sequence consisting of20 data points was performed. Using a math modelderived from this tare sequence eliminates the need toreposition the balance at each data point. The SVS


calibration required a total of three days to complete.The SVS experimental apparatus are shown in Figure 3.

Analyses of the data

The use of formal experimental design methodologyenables the application of numerous analyses anddiagnostics that have not been utilized for the review ofbalance calibration data. Many of these analysistechniques are valid only because of the way in whichthe experiment was conducted. This makes it difficultto directly compare the results from the OFATcalibration to the MDOE calibration. Also, these newanalyses are probably unfamiliar to those within thebalance community. Therefore, a mixture ofconventional calibration reporting methods and theMDOE methods will be presented. ConventionalOFAT calibration results include a comparison of themath model coefficients, plots of the residual errorsversus data point in the design, and a two-sigmaestimate of the residual errors.

There are many sophisticated methods of analyzing thecalibration data from the MDOE design. It is notwithin the scope of this paper to present all of thepossible analyses that are available. Rather, selectionshave been made that will highlight the new types ofinsight that can be provided to the balance engineer.These examples include the determination of theminimum number of model coefficients and an analysisof the unexplained variance.

Conventional OFAT experiment analyses

This section provides the results from the SVS, usingconventional reporting methods. The coefficients thatare generated by the mathematical model are not merelynumerical values; they have a physical meaning.Experienced balance engineers not only review themathematical model from the viewpoint of curvefitting, but also from the perspective of the physicsbehind the balance response. This knowledge of thebalance construction and physical performance isinvaluable in reviewing the math model.

LaRC force balances are of similar physicalconstruction and have typical interaction patterns thatare expected. Model coefficients are typically

expressed as a percent of full-scale effect. This enablesthem to be reviewed based on their relative magnitudeto each other. Figure 7 illustrates all of the coefficientsfrom the quadratic model for each balance response.There are 26 coefficients per component displayed oneach plot. The main effect, or sensitivity coefficient, isprovided in Table V and the intercepts are not shown.It can be seen that the coefficients derived from theMDOE calibration match closely with those from theOFAT calibration.

Table V reveals that the coefficients from the LaRCmanual calibration are consistently higher than thecoefficients from the SVS calibration. This differencewas investigated by performing a simple load sequenceusing the LaRC manual calibration hardware with thebalance connected to the SVS data acquisition system.Then, the same load sequence was performed using theSVS. The sensitivity coefficients derived from theseload sequences agreed within 0.01% of full-scaleresponse. This agreement, on the order of theresolution, eliminates the load apparatus as the sourceof the difference in the sensitivity coefficients. At thistime, it is suspected that instrumentation differencesbetween the data acquisition systems that were used forthe two calibrations are the source of the discrepancy.

At the resolution of Figure 7, it is difficult to discern theminor differences important to the level of precisionobtained from force balance measurements. Therefore,Figure 8 depicts the difference between the coefficients.The largest differences on these plots involve the PMterm on the YM response, and the NF term on the SFresponse, and are less than 0.20% of the full-scaleresponse. This difference is most likely due to differentlevel reference surfaces used during the LaRC manualcalibration. During the LaRC manual calibration, fivedifferent reference surfaces are used to define thebalance coordinate system. In the SVS, a singlereference surface is used, and the definition of theorthogonal axes is contained within the anglemeasurement device. The magnitude of these twoterms on Figure 7 reveals that the SVS coefficients aresmaller. A coefficient of zero would be theoreticallypredicted based on the balance design, and this wouldtend to point to the smaller coefficient as being correct.This qualitative analysis cannot be confirmed unless adetailed mechanical inspection is performed.

Table V. Sensitivity Coefficients, units are pounds or inch-pounds per (microvolt/volt).

Design Normal Axial Pitch Roll Yaw Side

OFAT 729-point 1.00206E-01 4.76009E-02 1.47139E-01 9.49683E-02 1.43170E-01 8.74642E-02

MDOE 124-point 1.00028E-01 4.75262E-02 1.46951E-01 9.48932E-02 1.42941E-01 8.73010E-02


Figure 7. Comparison of calibration model coefficients.

Effect of model term (percent of full-scale response) versus Model term.


Figure 8. Difference between model coefficients.

Delta effect of model term (percent of full-scale response) versus Model term.


A common plotting scheme that is used to review thequality of the fit of the math model is provided inFigures 9 and 10 for the OFAT 729-point and theMDOE 124-point designs. These plots illustrate thetotal residual error expressed as a percent of full-scaleload for each balance response. It is important toremember that the MDOE 124-point design involvesnumerous multiple component combinations ascompared to the OFAT 729-point design. Typically,the residual errors become considerably larger when themath model derived from the OFAT 729-point design isapplied to multiple component combinations. One ofthe factors that cause this situation is the difficulty inapplying orthogonal force vectors with the LaRCmanual system, as previously discussed.

Another factor is that the model coefficients are notderived from data that contains multiple componentcombinations. The OFAT design contains primarilyone and two component combinations. We know thatthe “real” balance calibration model is not quadratic, itis approximated as a quadratic polynomial. Significanthigher order terms are believed to exist, and theireffects must be absorbed into the available lower orderterms in the model. This is not necessarily desirable,but it is required to obtain the best possible model of aspecified degree. A technique is available to determinewhich higher order terms are absorbed into which lowerorder terms. It involves the computation of the aliasmatrix1 and provides excellent insight into thisphenomena. It is not within the scope of this paper toprovide the results from the alias matrix, simply topoint out that a method exists to investigate the designfor alias effects. The calibration design must providecombinations of the independent variables that includethese higher order terms for their effects to be absorbedinto the lower order terms. The MDOE design includesthese combinations, and therefore the fit of the modelon these multiple component combinations is better. Itis also important to note that multiple componentcombinations are more commonly encountered in thewind tunnel use of the force balance, and this is anotherbenefit of providing them in the calibration design.

A comparison of the two-sigma values for the totalresidual errors from the two calibration systems isprovided in Table VI. This is typically referred to asthe balance accuracy, but the potential errors in thiscalculation due to the lack of statistical independence ofthe data points within the LaRC OFAT design havebeen discussed. From this table, the high quality of thisparticular test balance is apparent since all of the valuesare less than or equal to 0.10% of full-scale load. Thesevalues also compare well between the two calibrationsystems. To put the resolution of these values inperspective, the weights used during the calibrationexperiments are accurate to within 0.01% of full-scalemagnitude. This means that simply using differentphysical weights during a single calibration could causedifferences on the order of 0.01% of full-scalemagnitude.

MDOE experiment analyses

As previously stated, the smallest model with thefewest parameters is desired since each coefficientcarries its own uncertainty. The objectives for thedetermination of the model coefficients are to provide amathematical model that has insignificant lack-of-fitand meets the precision requirement. An analysis ofvariance (ANOVA) was performed to achieve theseobjectives. One aspect of the ANOVA is a systematicmethod to determine the statistical significance of eachmodel coefficient (parameter). Table VII contains aportion of the data from the ANOVA of the normalforce response. In this table, all possible quadraticmodel terms are provided except the cross product ofaxial times roll. Due to the physics-based constraintpreviously discussed, only twenty-six terms can bemodeled simultaneously. It was determined that axialtimes roll had the least significant effect on the normalresponse from the three cross products in Equation 2,and therefore it was eliminated first. This set of data isused as an example of how a determination is made onwhich terms to retain in the model. The sameprocedure applies to the other five response variables.


Figure 9. Residual errors from OFAT 729-point calibration experiment.

Total residual (percent of full-scale load) versus Data point.


Figure 10. Residual errors from MDOE 124-point calibration experiment.

Total residual (percent of full-scale load) versus Data point.


Table VI. Comparison of two-sigma accuracies.

Residual Error expressed as two times the standard deviation (95% confidence)

(percent of full-scale load)

Design Normal Axial Pitch Roll Yaw Side

LaRC 729-point 0.03% 0.05% 0.05% 0.08% 0.04% 0.04%

MDOE 124-point 0.03% 0.05% 0.06% 0.10% 0.09% 0.07%

Table VII. Initial model for the normal force response.

Source DF Mean Square F-value Prob > FBlock 3 1.846E+05Model 26 3.272E+06 53,903,167 < 0.0001N 1 8.326E+07 1,371,566,591 < 0.0001A 1 3.013E-01 4.96 0.0283P 1 1.934E+03 31,867 < 0.0001R 1 2.993E+04 493,064 < 0.0001Y 1 1.002E+01 165.08 < 0.0001S 1 1.504E+02 2478.21 < 0.0001N2 1 5.204E-01 8.57 0.0043A2 1 4.337E-02 0.71 0.4001P2 1 1.647E-01 2.71 0.1028R2 1 6.019E-01 9.92 0.0022Y2 1 1.581E+00 26.04 < 0.0001S2 1 4.163E-01 6.86 0.0103NA 1 3.841E+00 63.27 < 0.0001NP 1 1.485E+00 24.46 < 0.0001NR 1 1.682E-02 0.28 0.5998NY 1 9.687E-04 0.02 0.8997NS 1 3.558E-02 0.59 0.4459AP 1 1.632E+02 2687.90 < 0.0001AY 1 1.476E-02 0.24 0.6231AS 1 1.250E-03 0.02 0.8862PR 1 2.173E-01 3.58 0.0616PY 1 8.436E-04 0.01 0.9064PS 1 1.845E-02 0.30 0.5827RY 1 4.322E+01 712.01 < 0.0001RS 1 3.435E+02 5658.28 < 0.0001YS 1 4.248E+00 69.97 < 0.0001Residual 94 6.070E-02Lack of Fit 86 6.123E-02 1.11 0.4785Pure Error 8 5.502E-02Total 123


In the left most column is the source of variance. Thesources of variance in this table include the explainedvariance (model terms and block effects) and theunexplained variance (lack-of-fit and pure-error). Thesecond column contains the number of degrees offreedom (DF) used to estimate the variance from eachsource. In the third column the variance, or meansquare error (MSE), of each model coefficient isprovided in units of microvolts per volt quotientsquared. The fourth column contains the F-value,which is equal to the ratio of the variance of eachcoefficient divided by the residual variance. The right-most column contains the probability that an F-valuethis large, with the associated degrees of freedom, couldhave occurred due to chance variations in the data(experimental noise). The smaller this probability, themore confidence that we have that the model coefficientis non-zero. For example, probability values of lessthan 0.05 suggest less than a 5% probability of a chanceoccurrence due to noise resulted in this regressioncoefficient. Stated another way, it is the probability of

getting an F-value of this size if the term did not havean effect on the response. If this probability is less thana threshold value then the coefficient is consideredsignificant and is retained in the model. Note the largeF-value and associated low probability of the N term onthe normal force response. This is expected since itrepresents the sensitivity of that particular componentand there is a strong correlation between the applicationof normal force and the associated normal forceresponse. The threshold probability value used for thisexperiment was 0.05. This means that there must begreater than a 95% probability that a coefficient is non-zero in order to retain it in the model. All coefficientsthat were below this probability were removed and thereduced model is shown in Table VIII. It is the goal ofthis phase to minimize the number of coefficients in themodel, because each coefficient carries its ownuncertainty. Once the model has been determined, thenan analysis of the unexplained variance can beperformed.

Table VIII. Reduced model of the normal force response.

Source DF Mean Square F-value Prob > FBlock 3 1.846E+05Model 16 5.317E+06 88,666,344 < 0.0001N 1 8.383E+07 1,397,975,758 < 0.0001A 1 3.164E-01 5.28 0.0236P 1 1.937E+03 32,308 < 0.0001R 1 3.021E+04 503,733 < 0.0001Y 1 1.013E+01 169 < 0.0001S 1 1.514E+02 2,525 < 0.0001N2 1 3.397E-01 5.66 0.0191R2 1 4.437E-01 7.40 0.0076Y2 1 2.657E+00 44.30 < 0.0001S2 1 2.847E-01 4.75 0.0316NA 1 3.748E+00 62.49 < 0.0001NP 1 1.719E+00 28.67 < 0.0001AP 1 1.641E+02 2,736.44 < 0.0001RY 1 4.367E+01 728.21 < 0.0001RS 1 3.494E+02 5,826.65 < 0.0001YS 1 4.763E+00 79.43 < 0.0001Residual 104 5.997E-02Lack of Fit 96 6.038E-02 1.10 0.4894Pure Error 8 5.502E-02Total 123


An analysis of the unexplained variance was performedin order to partition it into two components, lack-of-fitand pure-error. Typically, in the field of balancecalibration, the accuracy is based on the standarddeviation of the residual errors obtained by computingthe difference between the actual values and the modelpredicted values. This is normally expressed as twotimes the standard deviation providing a 95%confidence interval. As previously mentioned, theassumption of statistical independence of the datapoints from an OFAT calibration is not valid. Evenrelatively mild correlation can corrupt varianceestimates substantially, introducing significant errorsinto estimates of 95% confidence intervals. Moreimportantly, the total residual error includes twodistinct components, lack-of-fit and pure-error. Thelack-of-fit relates to the ability of the math model tocapture the response of the balance electrical signals asa function of the independent variables. The pure-erroris a function of the repeatability of the measurementenvironment. This includes factors such as thehardware used to set the independent variables, the dataacquisition system, the balance instrumentation, thequality of the mechanical interfaces, and the thermalstability of the calibration laboratory. For the purposeof this analysis, the repeatability of the calibrationsystem and the balance will not be separated. We alsoknow that we can not fit the data better than we canrepeat the data. MDOE provides a technique toseparate the lack-of-fit and pure-error components ofthe unexplained variance.

The pure-error component is computed from thegenuine replicates that are performed throughout thecalibration experiment. In the case of the MDOE 124-point design, there were three replicates per block,totaling twelve replicates in four blocks. Subtractingthe mean value of each block from these replicatesprovides eight degrees of freedom (DF) to estimate thepure-error. The sum of the squared (SS) deviationsfrom the means of each block is computed. The meansquare error (MSE), variance, can be computed basedon Equation 5.

MSESS

DFpure error

pure error

pure error

= (5)

Once the pure-error is known, its contribution to thetotal residual can be determined. This computationinvolves subtracting the SS of the pure-error from theSS of the total residual as shown in Equation 6.

SS SS SSlack of fit total residual pure error= − (6)

The MSE of all three quantities (total residual, lack-of-fit, and pure-error) can then be computed using theassociated degrees of freedom (DF) and the SSaccording to Equation 5. The ratio of the MSE of thelack-of-fit divided by the MSE of the pure-error formsthe F-value as shown below.

FMSE

MSEvalue

lack of fit

pure error

= (7)

This F-value is compared against a critical value of theF-distribution that depends on the degrees of freedomfor both lack-of-fit and pure-error and the specifiedsignificance of the test, 0.01 in our case. This 0.01significance level means that if our measured F-statisticexceeds the critical F-value, we can reject the nullhypothesis with 99% confidence. The null hypothesisin this case is as follows: Ho: The variance of the lack-of-fit is not significant relative to the variance of thepure-error. If the F-value is greater than the critical F-value then the null hypothesis can be rejected. In thiscase, it can be stated that we have 99% confidence thatthe lack-of-fit is significant. On the other hand, if theF-value is smaller then the critical F-value, then wewould not reject the null hypothesis, concluding that weare unable to detect significant lack-of-fit with ourrequired 99% level of confidence. This F-testprocedure provides an objective method fordetermining whether or not the model has significantlack-of-fit. Significant lack-of-fit means that thecalibration response function does not adequatelyrepresent the data upon which it is based.

A summary of the results of the above analysis isprovided in Table IX. This table includes the values ofthe quantities discussed in the partitioning of theunexplained variance. The columns contain theassociated values for each of the response variables.

The data in this table provides insight into themathematical model and the physical calibrationsystem. The sigma estimates in the table are computedaccording to Equation 8.

σ = MSE (8)

The repeatability of the measurement environment canreadily be determined from the sigma row of the pure-error section. For example, the standard deviation ofthe pure-error for the normal force response is 0.23


microvolts per volt. Comparing this to the full-scaleresponse (maximum response) of normal force providesa repeatability of 0.014% of full-scale. Thisrepeatability of 1.4 parts per 10,000 reveals the high-precision of the force balance and calibration systeminvolved in this experiment.

Insight is also provided by the results of the lack-of-fittest. For example, the normal force response does notindicate significant lack-of-fit. This means that addinghigher-order coefficients to model the normal forceresponse would not be justified, because the modelcurrently fits within the experimental error of themeasurements. Adding higher-order terms would onlyattempt to fit noise, which is undesirable for a usefulmodel. In classical balance modeling, higher-ordermodels or models with more coefficients have beenused to reduce the total residual error withoutconsideration to their justification. It is true that themore terms that are added to the model, the lower theresidual error, but what we desire is a model thatadequately represents the response of the balance, notthe noise. Conversely, the lack-of-fit is consideredsignificant on the rolling moment response, andtherefore a higher-order model could be used to providea better fit.

It is important to realize that the lack-of-fit test isrelative to the pure-error. In the limit, as the pure-errorgoes to zero, the lack-of-fit F-value goes to infinity.The decision to use a higher-order model is also tied tothe end use of the force balance. The real question inthe application of the force balance to wind tunneltesting is whether the prediction quality (precisionrequirement) and prediction risk (inference error risk)are sufficient to estimate the aerodynamic performancecoefficients on a scaled wind tunnel model.

In order to make this determination, the precisionrequirement and the inference error risk must beknown. Currently at LaRC, a force transducer isconsidered to be of the highest attainable quality, if theprecision is less than 0.10% of full-scale with a 99.15%probability that any one component is within theprecision requirement, meaning that there is no lessthan a 95% probability that all six components aresimultaneously within the precision requirement. Thebalance engineer must make a decision on the need forhigher order models based on the aerodynamic loadprediction requirements, not just on the pursuit ofproviding zero error in the balance measurement.

Table IX. Summary of the analyses of unexplained variance.

Quantity Normal Axial Pitch Roll Yaw Side

Maximum Response 1600 1052 1701 1054 875 917

Lack-of-Fit (SS) 5.7965 7.3089 26.1832 30.0050 15.2744 4.0042

Lack-of-Fit (DF) 96 93 100 101 101 99

Lack-of-Fit (MSE) 0.0604 0.0786 0.2618 0.2971 0.1512 0.0404

Lack-of-Fit (sigma) 0.25 0.28 0.51 0.55 0.39 0.20

Pure-Error (SS) 0.4402 0.7222 0.4480 0.0929 0.0859 0.2743

Pure-Error (DF) 8 8 8 8 8 8

Pure-Error (MSE) 0.0550 0.0903 0.0560 0.0116 0.0107 0.0343

Pure-Error (sigma) 0.23 0.30 0.24 0.11 0.10 0.19

Residual (SS) 6.2367 8.0311 26.6312 30.0768 15.3603 4.2785

Residual (DF) 104 101 108 109 109 107

Residual (MSE) 0.0600 0.0795 0.2466 0.2759 0.1409 0.0400

Residual (sigma) 0.24 0.28 0.50 0.53 0.38 0.20

measured F-value 1.10 0.87 4.68 25.57 14.08 1.18

critical F-value (@ a = 0.01) 4.97 4.97 4.96 4.96 4.96 4.96

Significant LoF (@ a = 0.01)? No No No Yes Yes No

units: maximum response is (microvolts/volt), SS is (microvolts/volt)^2, MSE is (microvolts/volt)^2, sigma is (microvolts/volt)


Future Research

Further research in the refinement of the SVS isactively progressing at NASA LaRC. Refinements tothe hardware system include a higher load range systemand automation. It is believed at this time that foursystems will be required to span the range of LaRCbalances, from ten pounds to ten thousand pounds ofnormal force. Automation of the system has beenconsidered in the areas of non-metric positioning, loadpoint positioning, and load application. Maintainingthe accuracy in the setting of the independent variableswill be paramount in all design decisions with regard toautomation. An automated system could reducecalibration to hours, instead of days. It would alsoprovide the ability to perform repeated calibrations andenable the application of statistical process controlmethods.

MDOE research efforts include higher-order models,and calibration at temperature. MDOE provides asystematic method for research into better mathematicalmodels. A cubic design has been generated and it isplanned to be executed on the test balance. It iscommon practice to include partitioned quadraticcoefficients in a balance math model to improvetroublesome coefficient fits. These partitionedquadratic coefficients, often referred to as split terms,are more likely higher order terms, such as cubic. Acomplete cubic model including pure cubic, quadratictimes linear, and all possible three-way interactions willprovide a more robust means to handle these higher-order terms.

It is generally known that balance calibration modelsare a function of temperature. At the present time, allLaRC complete balance calibrations are performed atroom temperature (70 degrees Fahrenheit). Fewbalances operate at room temperature in the wind tunnelenvironment. An abbreviated OFAT sequence of loadsis performed at elevated (170 degrees Fahrenheit) orcryogenic (-290 degrees Fahrenheit) temperaturedepending on the facility in which the balance will beused. The results of the abbreviated temperature loadsare difficult to interpret due to the inability to separatethe repeatability of the measurement environment fromthe actual thermal effects. A calibration design thatincorporates balance temperature as an independentvariable would provide thermally compensatedcalibration models.

Concluding Remarks

The experimental results from the Single-VectorBalance Calibration System have provided a proof-of-concept. This new integrated hardware system andcalibration design optimizes the calibration process.The benefits of this system and the application ofMDOE to balance calibration have been presented. TheSVS addresses the productivity limitations of currentforce balance calibration systems while simultaneouslyimproving the data quality. With the application ofMDOE, significantly fewer data points were generated,but the quality of the information that was obtained ishigher. The experimental results illustrate that acomplete six component calibration can be performedon the present SVS in three-days with nearly an orderof magnitude reduction in the number of data points.Formal experimental design techniques provideobjective methods to review and report the results ofthe calibration experiment. They also provide newinsights into force balance calibration that has not beenavailable with OFAT calibration experiments.

The SVS provides a significant advancement in forcebalance calibration technology. The insights that itprovides will aide in the advancement of other areas offorce balance design and production. Ultimately, thebenefit to the research community will be an increasedaccuracy of force and moment measurement duringwind tunnel testing.

Acknowledgements

The authors would like to thank Dick DeLoach for hispioneering efforts in the application of MDOE to windtunnel testing and his willingness to provide expertassistance during this research effort. We would alsolike to acknowledge the Langley Wind TunnelReinvestment Program who provided funding for thiseffort, and Tom Stokes and Ron Dupont of ModernMachine and Tool Company for their assistance in thedesign and fabrication of the prototype hardwaresystem.

References

1) Box, G.E.P.; Draper, N.: Empirical Model-Buildingand Response Surfaces. John Wiley & Sons. 1987.

2) Ferris, A. T.: Strain Gauge Balance Calibrationand Data Reduction at NASA Langley ResearchCenter. Paper DR-2, First International Symposiumon Strain Gauge Balances. October, 1996.


3) Levkovitch, Michael: Automatic BalanceCalibrator at Israel Aircraft Industries. January1993.

4) Polansky, L.; and Kutney, J. T. Sr.: A New WorkingAutomatic Calibration Machine For Wind TunnelInternal Force Balances. AIAA 93-2467. June1993.

5) Levkovitch, Michael: Accuracy Analysis of the IAIMark III ABCS. July 1995.

6) Parker, P.A.; and Rhew R.D.: A Study of AutomaticBalance Calibration System Capabilities. SecondInternational Symposium on Strain GaugeBalances. May 1999.

7) Marshall, R. and Landman, D.: An ImprovedMethod for Determining Pitch and Roll AnglesUsing Accelerometers. AIAA 2000-2384, June2000.

8) Beer, F.P.; Johnston, E.R. Jr.: Vector Mechanicsfor Engineers. McGraw-Hill. 1962.

9) DeLoach, R.: Applications of Modern ExperimentDesign to Wind Tunnel Testing at NASA LangleyResearch Center. AIAA 98-0713, 36th AerospaceSciences Meeting and Exhibit, Reno, NV. Jan.1998.

10) Dahiya, R.C.: Statistical Design for an EfficientCalibration of a Six Component Strain GageBalance. Old Dominion University ResearchFoundation, work performed under LaRC contractnumber NAS1-15648. August 1980.

11) Montgomery, D.C.: Design and Analysis ofExperiments, 4th edition. John Wiley & Sons.1997.

aiaa-2001-0170 a single-vector force calibration method ...mln/ltrs-pdfs/nasa-aiaa-2001-0170.pdf ·...

Documents