CAD Model-Based

Localization of Parts

in Manufacturing

Kristjan T. Gunnarsson and Friedrich B. Prinz

Carnegie Mellon University

We need automaticpart localization inflexible manufacturing.One method ofachievlng it uses a

CAD model andsparse data.

W e present a solution to theproblem of determining thelocation of a part in a robotic

manufacturing situation. The problem issolved by matching a database surfacedescription of the part with discrete pointsmeasured on its surface. The matching isaccomplished through the calculation of atransformation. The transformationspecifies the position and orientation ofthe part as it appears in front of the robot.The method was developed for use in flex-ible manufacturing where low precision,general purpose fixtures, and positionersare employed. Knowledge of the approx-imate location of the part is used. In a typi-cal application the uncertainty in thelocation of the part is small relative to thedimensions of the part, but more than anorder of magnitude greater than the toler-ances allowed by the manufacturingoperation.The introduction of a robot into a man-

ufacturing process requires an investmentin time and effort to program the robot.Consequently, industrial robots havefound use in large-batch manufacturingsuch as the automotive industry, where alarge volume of identical parts justifies theinitial outlays for programming. Profita-ble deployment of robots in small-batchmanufacturing demands a high degree of

flexibility with respect to the tasks therobot performs and adaptability tochanges in the robot's environment. How-ever, with flexibility comes uncertainty.The cost of precision fixtures precludesthem from small-batch manufacturing.The robot must be equipped with intelli-gence that allows it to adjust to the loca-tion of the parts in its environment.A human laborer can easily cope with

the randomness of the factory environ-ment. He will either pick up or approachthe parts wherever they may be. The partsare generally not placed precisely in thesame location every time, but randomlyplaced within a certain area. Adumb weld-ing robot will attempt to weld the parts inprecisely the same location every time, i.e.the programmed location. If the part is notprecisely at that location, the robot will notweld properly; if the part is missingaltogether, the robot will still attempt toweld, and will weld in thin air.A robot able to determine the location

of production parts in its working enve-lope significantly expands the scope oftasks and manufacturing situations inwhich it works. Set-up time and costs fora production run are reduced because thedesign and construction of precision fix-tures is eliminated. In general, the need totailor the environment precisely to the

00168-9162/87/0800-0066$01.0019871EEE66 COMPUTER

Figure 1. Definition of the world and CAD database (part) frames of reference,indicating an uncertainty in the location of a part. The ideal or planned location ofthe part is drawn in dotted lines, but an actual location is drawn in solid lines.

robot is relaxed because the robot canadapt to its environment.

Part localizationPart localization refers to the problem

of determining the position and orienta-tion of an object. Referring to Figure 1,localizing a part means being able toanswer the following question: What arethe Cartesian coordinates ofany specifiedpoint on the part in the robot's worldframe? In robotic manufacturing, the partlocation must be determined or assumedbefore an operation can be accomplished.In drilling, for example, holes cannot beplaced accurately unless the part's positionis known; long holes may be incorrectlyangled if the part's orientation is notknown.To answer this question, all points on

the part must be known. ACAD databasedescription of the surface of the part sup-plies this information. Associated with theCAD database description is a coordinateframe referred to as the CAD or partframe; this is the frame in which the parthas been designed and viewed on thescreen of a CAD station (see Figure 1).However, the location of the part, and theCAD frame, in the robot's world frame

depends on where the part is placed infront of the robot. The location of theCAD frame in the world frame is given bythe transformation, T1, which is repre-sented by a 4 x 4 homogeneous matrix ofthe form

F n, o° a, t,

T, a2 02 a2 t2 (1)nM3 03 a3 t30 O 0 1

The column vectors of T1, n, o, and a,specify the orientations of the CADframe's unit vectors in the world frame,and the column vector t specifies the ori-gin oftheCAD frame in the world frame.Ifx is some point on the part expressed inthe CAD frame, then T1x is the samepoint in the world frame.

If the location of the CAD frame isuncertain, points must be measured on thepart's surface as it appears in front of therobot. By mathematically matching theCAD database description ofthe part withthe measured points we can find the loca-tion of the part. The objective of thematching is to transform the CAD frameinto the robot's world frame and therebydetermine TI. The only informationneeded from the database are the equa-tions of the surface of the part. A CAD

database may contain information aboutthe edges, vertices, and volumes of a part,but we don't need this informationbecause no attempt is made to measureedges, vertices, etc. In summary, localiza-tion requires a CAD database surfacedescription of the part, measurement ofsurface points on the actual part, andmatching the two.The database description is assumed

complete in the sense that every point onthe part's surface is accounted for. How-ever, a part can be localized with only sixmeasured points. The measured data isconsidered sparse when 6 to, say, 60 pointsare measured. This is frequently the caseif tactile position sensing is used. However,when machine vision and laser ranging sys-tems gather hundreds of data points, thedata is called dense. Many optical systemsare highly suitable for part localizationapplications because of their speed andaccuracy, and dense data can be reducedto sparse data. The method described inthis article was developed for sparse data,but is equally suitable for tactile and non-tactile (optical and ultrasonic) sensing. Forexample, a simple tactile sensor used forpart localization uses electrical contactbetween the electrode of an arc weldingtorch and the workpiece.

Part localization has primarily beenstudied by researchers working in the areasof machine vision and scene analysis.'Although these technologies are not neces-sarily intended for application in manufac-turing, they have nevertheless found usethere. The two principal objectives in sceneanalysis are recognition of the objectsmaking up the scene and composition ofa general description of the scene.2Accurate localization of the objects in thescene is a secondary consideration in manyinstances. In manufacturing applicationsof machine vision, part localization isimportant and thus has received consider-able attention.3 Part localization methodsdeveloped by machine vision scientists arebased primarily on the use of dense data,because the nature of vision sensing is togather large amounts of data.4'5 Sparsedata methods invented by visionscientists6 usually include the reduction ofthe original dense data as a first stage.The method described in this article

differs in many respects from machinevision methods. A primary function ofscene analysis algorithms is identificationand recognition of the constituent objectsof the scene. Here, we assume that the partto be localized is known. This assumptionis iustified in most manufacturing situa-

August 1987 67

tions. The emphasis in our method is onaccurate localization and sparse data. Themethod does not require estimation of sur-face normals nor calculation of the surfaceof the part.

Applications ofpart localization

Part localization is needed in variousrobotic manufacturing situations. Inmechanical assembly, tight-fitting partscannot be assembled unless their relativepositions and orientations are known. Suc-cessful grasping depends on knowing theaccurate orientation of a part prior toattempting the grasp. Mechanical compli-ance only reduces the needed localizationaccuracy. In robotic arc welding, forexample, the welding torch must be posi-tioned accurately in the weld groove. Ingas-metal-arc welding (GMAW), thecross-seam positioning uncertainty mustnot be worse than the diameter of the elec-trode. Normal to the seam, the accuracymust be even greater. Failure to accuratelyposition the torch produces a defectiveweldment and may possibly damage ordestroy the welding torch. Gauge-freeinspection directly measures key points onthe surface of a manufactured part, tocheck dimensional integrity as specified bythe design. The part is matched with a pro-totype or a database by measuring cor-responding points, both on the part and inthe reference description. However, thepart location must be known before aprescribed point on the part can bemeasured.The general solution to the localization

problem is to avoid it by use of dedicatedfixtures or specialized machinery. Thissolution is not economically justified insmall-batch and flexible manufacturing,where general-purpose fixtures are used.The greater flexibility of such fixtures isobtained at the expense of accuracy; partsof various shapes and sizes can be fixed inthem, but none can be positionedaccurately.

Part-localization methods have animportant application in connection withmodel-based off-line programming ofrobots. The alternative to off-line pro-gramming is teach-by-doing, where therobot arm is physically led through themotions of the task and points along itstrajectory are stored in the controller'smemory. Robots are used to their bestadvantage when programmed off-line,

irrespective of the manufacturing task, beit part handling, welding, or inspection.Unlike the teach-by-doing mode, model-based off-line programming does notrequire the physical use of a robot andpart, and production is not disrupted.With off-line programming, aCAD data-base description of the part is used to planthe motion of the robot. For instance, inthe case of flame cutting or deburring, thepath over which the robot must carry thetool and its orientation are calculated fromthe database description of the part's sur-face. A kinematic model is then used tocalculate the joint motions ofthe robot, orthe robot is programmed directly in worldcoordinates.

Part localization is essential for off-lineprogramming to be effective. The com-

Robots are used totheir best advantagewhen programmed

off-line.

plete tool path and orientation relative tothe part are calculated off-line. However,the position and orientation of the path asa whole depend on the actual location ofthe part as it appears on the factory floorin front of the robot. This situation isdemonstrated in Figure 1. The part, andconsequently the tool path and tool orien-tation, is described in the CAD frame,which is independent of the part's actuallocation, but the robot trajectory dependson the actual location of the part in itsworld frame.Assume that a task is specified by a

sequence of Cartesian points, xi, along thesurface of a part; these are the points therobot tool must follow to carry out thetask. If these points have been generatedat aCAD station using a model of the part,then the trajectory points are specified inthe CAD frame. (Notice that this taskdescription is independent of the part'sactual location.) Normally, the parts areexpected to appear in front of the robot ina planned location called the ideal locationand specified by the transformation T,(see Figure 1).

If a part now appears in the ideal loca-tion, then what are the robot's trajectorypoints for the specified task? To answer

this question, the points in the CAD framemust be transformed, or projected, intothe robot's world frame:

xwv = TIXCADi (2)

where XCAD. are the task points specifiedin the CAIJ frame, and xw are the cor-

responding robot trajectory'points in therobot's world frame.

In manufacturing situations where thelocation of a part is uncertain, the preced-ing procedure is likely to be inaccuratebecause the ideal location transformationT1 may not refer to the actual part loca-tion. Referring to Figure 1, a part is shownin a location differing from the ideal loca-tion by the offset transformation T0ff. Inthis case, the true location transformationis given by the multiple ofthe ideal and off-set transformations, and the robot trajec-tory points are

Xwi = T,ToffXCADi (3)

The part localization method presented inthis article determines the true location(transformation) of a part using its CADdatabase description and discrete mea-sured points on the actual part.The method just presented gives a robot

the ability to adjust to different locationsof the parts in its working envelope. Notethat this procedure is not limited to model-based off-line programming. In fact, it isequally applicable to the teach-by-doingmethod. In this case, a robot has beentaught a trajectory for a part in a specificlocation. If the next part is in a differentlocation, its true location is found by alocalization algorithm and the originaltrajectory is modified in a way similar tothat shown above.

Localization ofpolyhedra

This section presents a solution forlocalization of a part with a surface madeup of planar surface facets or patches. Pastresearch has devoted considerable atten-tion to such surfaces, commonly encoun-tered in design and frequently used tomodel or approximate surfaces of modestcurvature.

Previous methods of localization havenot only required the measurement of sur-face points, but also the local estimationof surface normals.3'6 Local estimation ofsurface normals on a surface requires that

68 COMPUTER

three points (or more) on a planar facet bemeasured. The local normal is calculatedby passing a plane through the points; thenormal of that plane becomes the localestimate of the surface normal. Althoughsurface points must be measured on a partto indicate its location, local surface nor-mal estimation is not necessary.

It is desirable to have a method of locali-zation that does not require local surfacenormal estimation. Local surface normalestimation is sensitive to random surfaceroughness and measurement uncertainties,which can lead to imprecise localization,but high localization accuracy is ofmajorimportance in most manufacturing appli-cations.

Accurate surface normal estimationrequires that the points used in calculatingthe normal be widely spaced over the sur-face facet to minimize the sensitivity toerrors. Many points clustered together,which are readily obtained with opticalsensing systems, are unsuitable. Whenonly the approximate location of a part isknown, reliable measurement of widelyspaced points on a specific planar facet isdifficult. Furthermore, measured pointsare often at a premium in manufacturingsituations. This is especially true if tactilesensing is used. It must be possible to local-ize a part with a minimum number ofpoints.We show a method for the localization

of arbitrarily-shaped surfaces in a later sec-tion. This method is based on repeatedpolyhedral approximations of the curvedsurface. While the measurement of thesurface normal of a plane is elementary inprinciple, the local estimation of the nor-mal of a curved surface is not. Therefore,it is desirable to have a localization methodthat does not require surface normals.

Problem formulation. We will not pre-sent the mathematical details of localizinga part by means of a rigid body transfor-mation here.5 Instead we discuss the basicideas and formulations of the procedure.Assume that a polyhedron must be

localized. Its CAD database descriptiongives the equations of all of its planarfacets, i.e. the set of m planar surfacefacets, S,(n3x1, dx1) i = 1, 2, 3, . . ., m.Each facet, Si, is characterized by its unitsurface normal vector, ni, and the short-est distance from the facet to the origin di,which is a scalar. This is the standard vec-torial representation of an unboundedplane. The normal vectors and shortestdistances (ni and d,) are given in the CAD

P2

RxF

b

aI

a

x

a

Figure 2. Illustrtion of the rotation and translation of a plane (shown in twodimensions) and the effect of these operations on the describing equation, where"a'" is the untransformed state, "b" is the rotated state, and "c" is the rotated andtranslated state.

frame. Points are measured on the surfaceof an actual part either by a robot-mounted, or a remote, sensor, and a set ofdiscrete points, xj1j = 1, 29 .... mi, isgenerated. These points are given in therobot's world frame. The Cartesian point(vector) xj, denotes thejth point measuredon the ith surface facet, and mi is thenumber of points measured on the ithfacet. Note that the correspondence ofevery measured point to a specific facet isassumed known. Localizing the part thenmeans finding a transformation thatmatches the planar surface facets withtheir corresponding points.

In general, if a plane and a point aregiven it is a simple matter to check if thepoint lies in the plane. This is done byinserting the point into the equation for theplane:

6 = n x - d (4)

Expression 4 gives the distance, 6, of thepoint x from the given plane; if the pointis in the plane, its distance to it is zero. Amatching between the surface and the

measured points can now be defined:Match the surface with the measuredpoints by making the distances between thepoints and their corresponding surfacefacets zero, or as small as possible. This isa very natural definition of a matching.The mechanical task of inserting a peg ina hole can be viewed as the task ofminimizing the distance between the sur-faces of the hole and peg, respectively.A matching has now been defined, but

what instrument is available for affectingthis matching? The answer is the localiza-tion transformation. The distancesbetween the measured points and the sur-face will be made as small as possible byappropriately rotating and translating theCAD database surface. With reference toFigure 1, the localization method mustanswer the question of where to positionthe CAD frame in the robot's worldframe, and how to orient it.A rigid body transformation is com-

posed of two parts: rotation and transla-tion. The translation is a vector (the fourthcolumn vector of the matrix in Expression1), but the rotation operator is a matrix

August 1987 69

(the upper left-hand 3 x 3 submatrix ofExpression 1). Figure 2 illustrates themathematical operation of rotating andtranslating a plane in two dimensions; intwo dimensions a planar facet becomes aline segment. Assume that the line segmenta shown in Figure 2 must pass through thepoints PI and P2 also shown in the figure;the line segment must be made to passthrough the points by rotating and trans-lating it. Denote the rotation matrix by Rand the translation vector by t. The effectof rotating segment a, S(n, d), is therotated segment b, S(R n, d). The rotationchanges the orientation of the line segment(the normal vector), but has no effect onthe shortest distance to the origin (asshown in the figure). The effect of trans-lating the rotated line segment is the seg-ment c, S(R n, d + (R n)t). Thetranslation has no effect on the segment'sorientation; it only changes its distance tothe origin.

Expression 4 is the distance of a point xfrom its untransformed line segment. Thedistance after the transformation is

6, = (R n)x - ((R n)t + d) (5)

In the general three-dimensional case, thedistance of a point xj from its cor-responding rotated and t'ranslated plane,S(R ni, di + R nit), is

6j, = (R ni)xji - ((R nd)t + d,) (6)

The rotation and translation must be sochosen to make the above expression zero(or as small as possible) for all points (i's)and surface facets (i's). Notice that theentire surface is transformed as a whole,i.e. all of its facets undergo the same rota-tion and translation. This leads to the leastsquares minimization problem,

m miminimize 2 ((Rni)xj,- (Rni)t - di)2 (7)

i= lji= I

The squared expression is the distance ofeach point to its respective facet. These dis-tances are summed over all points on allplanes, and the total must be minimized byproperly calculating the transformation (Rand t). The individual distances aresquared so that positive and negative dis-tances do not cancel out.

There are two major reasons for for-mulating the problem as an optimizationproblem. First, in this way, one problemformulation covers all possible situationsregarding both the total number and dis-tributions of the measured points on the

facets. Second, the effect of possible noisein the measured points can be reduced bymeasuring more than the minimum num-ber of points and overconstraining theproblem. In the case of perfect measure-ments and perfect agreement between theCAD database and the part, all distancesand their squared sum will be zero whenthe transformation has been correctly cal-culated. If errors exist, an optimal fitbetween the measured data and the data-base will be calculated. The numericalsolution is calculated by an iterative pro-cedure based on Lagrange multiplier andNewton-Raphson methods.5The above problem formulation places

no upper bound on the number of facets,and places no special conditions on the dis-crete points. However, every mathemati-cal problem must be properly formulated.

The most generalCAD database

description of a surfaceor part is in terms of

discrete points.

In the present case, the given surface andpoints must allow for the calculation of aunique transformation. Two basic condi-tions must always be met:

* A total of at least six points, measuredon at least three facets, must be provided.

* Any three facets for which at least onepoint was measured on each of them musthave surface normals that are linearlyindependent.At least six points must be measuredbecause a rigid body transformation ischaracterized by six independent variables:three degrees of translation (x, y, and z),and three angles of rotation (the three eulerangles, or the roll, pitch, and yaw angles).If points are measured on only threefacets, the only acceptable distributions ofexactly six points on those surfaces areeither 2:2:2 or 3:2:1. If only one point ismeasured on each facet, at least six facetsare required. If these conditions are notmet, a mathematical singularity is encoun-tered and a solution cannot be calculated.Furthermore, notice that the formulationdoes not assume the possibility of estimat-ing local surface normals from the senseddata.The correspondence of measured points

to specific surface facets is a basic assump-

tion of the algorithm. If a polyhedron hasn facets and k arbitrary points are mea-sured on it, the total number of combina-tions relating the points to the facets is n1.Assuming six facets and six points, thisnumber is 46,656! Grimson and Lozano-Perez6 considered all possible combina-tions, but presented powerful geometricpruning methods for eliminating impossi-ble point-facet combinations. However,their method requires surface normal esti-mation. The correspondence assumptionis justified when an estimate of the part'strue location is available. If this estimateis accurate to within a fraction ofthe part'sdimensions, the correspondence of mea-sured points to facets can be guaranteed.This is the case in the manufacturing appli-cations for which this method wasdeveloped.

Demonstration. The localizationmethod is demonstrated in Figure 3, whichshows the localization of a part made upof six planar facets. In the figure, the partdrawn in light lines represents the CADdatabase surface in the ideal location.Instead of drawing discrete points (sixpoints were used in this demonstration,one on each surface) representing theactual location of the part, another part isdrawn in heavy lines for the same purpose.This helps the observer, who otherwisewould have great difficulty visualizing theprogress of the algorithm. The numericalprocedure is both reliable and fast; typicalexecution time for this procedure on aVAX 11/750 is on the order of a second.The accuracy of the localization method

is primarily governed by the accuracy ofthe sensing system. As a rule ofthumb, wecan expect the localization inaccuracy tobe on the same order as the combined aver-age surface roughness and sensor uncer-tainty. The selection of measurementpoints is also important. Whenever possi-ble, points should be measured on thosefacets that have the greatest angular sepa-ration, i.e. facets whose normals areclosest to 90 degrees apart. This maximizesthe computational sensitivity and conse-quently the accuracy. The points shouldalso be spread over the surface to facilitateaccurate orientation estimation.

Localization ofquadric surfacesThe method described in the preceding

section has been extended to allow the

COMPUTER70

localization of surfaces comprised ofquadric patches. Quadric surfaces deservespecial attention for a number of reasons.These surfaces include planes, cones,cylindrical and elliptical pipes, spheres,etc., and are described by a second-orderimplicit equation in analytical geometry.They are common in mechanical designand thus frequently encountered in locali-zation problems. Surfaces represented byimplicit equations of order greater thantwo are difficult for a designer to visual-ize. Consequently, explicit parametricequations such as splines are used whendesigning a surface of order higher thantwo.

Figure 4 demonstrates this method,showing the localization of a part made upoftwo intersecting pipes. The part drawnin light lines represents the CAD databasesurface in the ideal location, and the partdrawn in heavy lines represents the actuallocation of the part. Six points were usedin this demonstration, three on each pipe.Again, the algorithm is seen to accuratelyconverge in few iterations.

Localization ofparametric surfacesThe basic problem to solve when localiz-

ing a surface is to match points measuredon a surface to a mathematicalCAD data-base description of that surface. Thematching procedure is to reduce the dis-tances between the points and the surfaceas much as possible. The most elementarymathematical operation in a matchingalgorithm calculates the distance betweena point and the surface. This makesimplicit equations the ideal database rep-resentation because they take the form

fix, y, z) = 0

Ib'

Figure 3. Demonstration of the localiza-tion algorithm for polyhedra. The parthas six facets, and we assume one datapoint measured on each facet. TheCAD database surface is drawn in lightlines in the ideal location; the part isdrawn in heavy lines in the actuallocation.

Figure 4. Demonstration of the localiza-tion of a part made up of two intersect-ing pipes. The CAD database surface isdrawn in light lines in the ideal location;the part is drawn in heavy lines in theactual location.

(8)

Simply by inserting the Cartesian coor-dinates ofa point, (x, y, z), into the implicitequation we can obtain a measure of thedistance between the point and the surface.The most general CAD database

description of a surface or part is in termsofdiscrete points, because all other surfacedescriptions can be reduced to that form.However, for localization purposes a data-base description must be complete in thesense that any point measured on an actualsurface must be accounted for in thedescription. This can be accomplished byuse of interpolating functions, such assplines, which are explicit representations

of a surface. The Cartesian coordinates ofany point on the surface are given by theequations

x = fi(u, v) (9)

y =f2(u, v) (10)

Z =f3(U, V) (11)

where u and v are independent parametricvariables, generally referred to as surfacecoordinates. Various kinds of splines areavailable, but for localization, the simplest

ones are bicubic parametric splines:

3 3

x(u,v) = E; Ia.u'ivji=Oj=O

(12)

where u and v are the curvilinear surfacecoordinates, aij contains the splineparameters, and x is any point on thesurface.

Explicit surface equations map a pointin a two-dimensional surface space to athree-dimensional Cartesian space:

P2(U, v)-*P3(x, y, z)

August 1987

(13)

71

When attempting to match a CAD data- to calculate the distance of a point to a expression for the distance. In the case ofbase expressed by explicit equations with parametric surface; we must use an itera- parametric surface descriptions, no suchmeasured points, a discrepancy in spaces tive numerical procedure. The localization expression can in general be derived.becomes obvious: the database is in a sur- method for polyhedra requires us not only The localization method presentedface space, but the measured points are in to calculate the distance of a point to the below is an iterative scheme based ona Cartesian (world) space. It is not trivial surface, but also requires an analytical repeatedly approximating the splined sur-

/I

a AThANFOMATION MDEFOUND. SUCHTHIATanaMswnDWTCRVB

REULTANMTRANSFOMATIONO51e voNmuoNnms.MaAow

b IWL- FNDhSORiIown'uIor1 f b"m Ib DFANaTOnMCLJVE

/

flW2S ATMO)UMT5 FECtRVE WYT ISIRAHTILATThEMPODMTOPSHORTDTDffrANM

g Immisa

- w - l @ l @ l li

/

d Srw3: INDThEOFMALTRANSFORMAI1ON OFTHEPOINT BASEDON THELNEARIZATION

h _ud3

Figure 5. Two-dimensional demonstration of the localization algorithm for parametrically described curves, showing the

localization of a cubic spline.

/

mI i 0 i It

-~ ~ ~0-

0a

a0

1

0

72 COMPUTER

face with a polyhedron. At the start ofeach iteration, the algorithm finds thepoints of shortest distance to the splinedsurface from each measured point. Thespline is approximated by planes (or, alter-natively, quadrics) at each such point. Atransformation is calculated based onthese approximations and the measuredpoints using the method for localizingpolyhedra. The CAD database surface isthen transformed closer to the measuredpoints. This procedure is repeated until asatisfactorily accurate match has beenreached. With this approach, the curvedsurface to be localized is approximated byan open polyhedron; this approximationis dynamically adjusted as the surface andthe measured points are moved closertogether. This method applies to both con-cave and convex surfaces.The method is summarized as follows:

(1) Assume the CAD database surfaceis in the ideal location.

(2) For each measured point, calculatethe point of shortest distance on thesurface.

(3) Approximate the CAD database sur-face with planes at the points of shortestdistance.

(4) Calculate an optimal transformationbetween the approximated surface and themeasured points using the method forlocalizing polyhedra.

(5) Apply the calculated transformationto the CAD database surface.

(6) Test for convergence. If the transfor-mation calculated in Step 4 is sufficientlyclose to the identity transformation, thenquit; else go back to Step 2.

This method is similar to Newton'smethod for calculating the roots of func-tions and shares many of its convergensecharacteristics.5 Figure 5 demonstratesthe method in two dimensions.

Figure 5a shows a curve segment (data-base description) and seven discrete points(measured points). The problem is to finda transformation to match the curve andpoints. In this demonstration the transfor-mation is applied to the points instead ofthe curve (database); this is a minor mod-ification, but computationally simpler. Asoutlined above, the first step is to find thepoints on the curve of shortest distance tothe measured points; Figure Sb shows theresults of this operation. The second stepinvolves linearizing the curve about thepoints of shortest distance, as illustratedin Figure Sc and Figure 5d, where theapproximating polygon is drawn together

Figure 6. Localization of a surface described by parametric equations (bicubicsplines). The CAD database surface is drawn in heavy lines in the ideal location; itis drawn in light lines in the actual location.

with the measured points. The third stepinvolves finding the optimum transforma-tion of the points based on the lineariza-tion; the results ofthis transformation areshown in Figures 5e and 5f. Note that thisprocedure converges in one iteration whenthe surface is a polyhedron. As seen in Fig-ure 5f, the points have moved very close tothe spline despite the large initial offsets.Figures 5g and 5h show how the wholeprocess is repeated twice, with a satisfac-tory match obtained in only three iter-ations.

Figure 6 shows a three-dimensionaldemonstration of this method. Thehemispheroidal surface is reminiscent of acanopy. This type of surface is difficult torepresent with implicit equations; it ishighly curved in certain places, but nearlyplanar in others. Figure 6a shows the sur-face in light lines rotated by about 10degrees about all three axes and translatedabout 1/5 of its scale from the ideal loca-tion (heavy lines). Eight measured points,widely distributed over the surface, wereused in the simulation, six points aroundthe sides and two on the top. Convergencetook place quickly, which is typical of thismethod.

This localization method is particularlyeffective against translation offsets.Translations affect the surface uniformly;all points are displaced equally. Rotations,on the other hand, do not displace all sur-face points equally; the greater the radiusof gyration of a point (distance from theaxis of rotation), the more it is displaced.

The method requires an initial estimate ofthe actual location ofthe surface (the ideal,or planned, location), which is possible inmost manufacturing situations. Specifi-cally, the method is intended for localiza-tion of parts in low-precisiongeneral-purpose fixtures, for jigs, and onpositioners, for which such information isreadily obtainable.

W* r e have demonstrated a methodof part localization designedfor manufacturing applica-

tions. This method is based on matchinga CAD database description of the part'ssurface with data measured on an actualsurface. The method is suitable for usewith both tactile and nontactile sensingsystems, and for both simple surfaces(polyhedra and quadrics) and arbitrarily-shaped surfaces. The method uses knowl-edge about the approximate location ofthepart prior to precise localization. Itrequires neither surface normal data northe determination of the equation of thesurface from the data. Instead, we obtaina direct matching through a rigid bodytransformation between the data anddatabase.The major manufacturing advantages

of part localization follow:* Part localization allows the use of

low-precision, but general-purpose, fix-tures needed in flexible and small-batchmanufacturing. It enables a robot to adaptto variations in the location of parts in its

August 1987 73

working envelope.* The design and construction of preci-

sion jigs and fixtures is time consumingand expensive, and localization methodsoffer potential savings in time and cost.

* Methods of automatic localizationenable gauge-free inspection of parts.

* Part-localization methods provide acritical link in the integration ofCAD/CAM with robotics. They bridgethe gap between the CAD station and thefactory floor in model-based off-line pro-gramming of robots. [1

AcknowledgmentThe research presented in this article was sup-

ported by the Welding Consortium, TheRobotics Institute, Carnegie Mellon University.

References1. R.C. Bolles and P. Horaud, "3DPO: A

Three-Dimensional Part Orientation Sys-tem," Int? J. Robotics Research, Fall 1986,pp. 3-26.

2. K. Ikeuchi, "Generating an InterpretationTree From a CAD Model to RepresentObject Configurations for Bin-Picking

Tasks," Tech. report CMU-CS-86-144,Dept. of Computer Science, Carnegie Mel-lon Univ., August 1986.

3. O.D. Faugeras and M. Hebert, "The Rep-resentation, Recognition, and Locating of3D Objects," Int'l J. Robotics Research,Fall 1986, pp. 27-52.

4. M. Brady et al., "Describing Surfaces,"Computer Vision, Graphics, and ImageProcessing, Vol. 32, 1985, pp. 1-28.

Kristjan T. Gunnarsson is a research engineerin the Department of Mechanical Engineeringat Carnegie Mellon University. His researchactivities include automatic tool calibration forrobots, automated inspection ofaluminum cast-ings, and off-line programming of robots.Gunnarsson received a bachelor's degree

from the University of Iceland in 1979, amaster's degree from Pennsylvania State Uni-versity in 1981, and aPhD degree from CarnegieMellon University in 1987, all in mechanicalengineering.

5. K.T. Gunnarsson, OptimalPartLocaliza-tion byDatabaseMatching With Sparse andDense Data, PhD dissertation, Dept. ofMechanical Engineering, Carnegie MellonUniv., May 1987.

6. W.E. Grimson and T. Lozano-Perez,"Model-Based Recognition and Localiza-tion from Sparse Range or Tactile Data, "Int'l J. Robotics Research, Fall 1984, pp.3-35.

Friedrich B. Prinz is a professor in the Depart-ment of Mechanical Engineering at CarnegieMellon University. He conducts research in theareas of robotics, CAD/CAM, and the appli-cation of artificial intelligence for manufac-turing.

Prinz received his PhD in physics from theUniversity of Vienna in 1975.

Readers may write to Prinz at the Departmentof Mechanical Engineering, Scaife Hall, Rm.403, Carnegie Mellon University, Pittsburgh,PA 15213.

Sofiwre EngineersNorthrop Corporation's Defense Systems Division in RollingMeadows, Illinois is the fastest-growing enterprise in an expan-ding electronic countermeasures industry. We offer professionalswith a BSCS, BSEE, BS Math or Physics (or equivalent) MS prefer-red, and a minimum of 3 years experience, opportunities in thefollowing areas.ManaemenI SyemAnirftec4 TdchnalLeade,sand engineeing assignments avaIabe.System ProgrammersOur many varied applications require significant growth in oursupport capabilities. We need the best people with experience in:* LANGUAGES, including Ada, Assembler, C, FORTRAN,JOVIAL, and Pascal

* OPERATING SYSTEMS, including UNIX and VMS* Development of Real-Time Operating Systems* Development of Software Tools* Performance Modeling and Evaluation* Use of Software Structured Development MethodologiesSoftware Systems EngineersOur software engineers develop software from system re-quirements through implementation, and need experience in:* Software Requirements Analysis * Architectural Design* Software Validation and Test Specification* Performance Specification and Modeling* Interface Design and SpecificationECM/EW Systems Software EngineersECM/EW Systems are our business. We need the best people withexperience in:* Real-Time Control Systems * Object Discrimination &* Radar Data Processing Classification

* Embedded Computer * ECM AlgorithmSystems Development

* System and Unit Level * Calman FilteringDiagnostics * Optimal Control

Hardware Diagnostics Software EngineersWe design and develop advanced systems using the latest hard-ware and software technologies for our military clients. Experiencerequired:* Intelligent Control Panel Systems * Built-in-Test* Development * Functional Test* Micro and Macro Diagnostics for Fault Identification

Arificial IntelligenceArtificial intelligence technologies promise state-of-the-art solu-tions to complex ECM/EW challenges. Pbsitions require peoplewho can bring Al technologies to avionic electronics, with ex-perience in:* LANGUAGES, including: Ada, C, LISP, and Prolog* System Prototyping * Knowledge Engineering* Implementation of Al Technology * Expert Systemsin Real-Time Embedded Systems Development

Interested individuals are encouraged to forward resume to: lameshascona, Technical Recnriter, Dept. C91, Nortrp Corporatin,Defense Systems Division, 600 Hicks Road, Rogling Meadows,IL 60008. We are an equal opportunity employer M/F/VIH. U.S.Citizenship Required.

NORTHROPDefense Systems DivisionElectronics Systems Group

.1. 40. Idopp, I .._Jllllmppp, ..w0010"