AD-A096 187 TELEDYNE BROWN ENGINEERING HUNTSVILLE ALA SYSTEMS DIV F/G 9/2SOFTWARE PARTITIONING SCHEMES FOR ADVANCED SIMULATION COMPUTER --ETC(U)FEB 81 S J CLYMER F33615-78-C 0013

UNCLASEHLTR0-42PT1

1 .0 12.0

11111 1.2.5 1j111 j . 6

MI( O( ROY RISOLUTION I SI CHARI

NAITONAL BURIAL ) SIANDARDS 1963 A

4,1 AFHR41 42 7 T- '/1a

AIR FORCE 1 0A3OFTWARE TARTITIONINGWHEMES FOR ADVANCED / _H SIMULATION COMPUTE R SYSTE MS .

U _ By

0S. I merSaye s uson

Teledyne Brown Eagineering300 Sparkman DriveA Huntsville. Alabama 35807

Nt.OPE RATIONS TRAINING DIVISIONWilliams Air Force Base, Arizona 85224

00 J PIFinE -Fina j Rept, ~ 9

? .. 3: !-' -

Approved for public rlea.: dilliuiion unlimiled.

U/

E LABORATORYTS

V AIR FORCE SYSTEMS COMMANDBROOKS AIR FORCE BASETEXAS 78235

81 3 17-0 2

NOTICE

When U.S. Government drawings, specifications. or other data are used for any purpose otherthan a definitely related Government procurement operation, the Government thereby incursno responsibility nor any obligation whatsoever, and the fact that the Government may haveformulated, furnished, or in any way supplied the said drawings, specifications. or other datais not to be regarded by implication or otherwise, as in any manner licensing the holder or anyother person or corporation. or conveying any rights or permission to manufacture, use. or sellany patented invention that may in any way be related thereto.

This final report was submitted by Systems Division. Teledyne Brown Engineering. 300Sparkman Drive, Huntsville, Alabama 35807. under Contract F33615-78-C-0013. Project6114, with the Operations Training Division. Air Force Human Resources Laboratory (AFSC).Williams Air Force Base. Arizona 85224. Pat Price was the Contract Monitor for theLaboratory.

This report has been reviewed by the Office of Public Affairs (PA) and is releasable to theNational Technical Information Service (NTIS). At NTIS. it will be available to the generalpublic, including foreign nations.

This technical report has been reviewed and is approved for publication.

MARTY R. ROCKWAY, Technical DirectorOperations Training Division

I:RONALD W. TERRY, Colonel. USAFCommander

-p * ~ . '.

UnclassifiedSECURITY CLASSIFICATOM OF ThIS PAGE (Slhm Dua nte see

REPORT DOCUMENTATION PAGE READ INSTRUCTIONSBEFORE COMPLETING FORM

I. RElPORiT NUM§R2. GOVT ACCESSION NO: S. RECIPIENT*S CATALOG NUMBERAFHRL-TR-8042 (Part ) r

4. TITLE (Ua"d SuIitt) S. TYPE OF REPORT & PERIOO COVERED

SOFTWARE PARTITIONING SCHEMES FOR ADVANCED FinalSIMULATION COMPUTER SYSTEMS

6. PERFORMING ORG. REPORT NUMBER

7AUTHOR(o) S. CONTRACT ON GRANT NUMBER(s)

S.J. ClymerF33615-78-C-0013 '

9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASKSystems Division. Teledyne Brown Engineering AREA A WORK UNIT NUMBERS

300 Sparkman Drive 62205FHuntsville. Alabama 35807 61142M4

It. CONTROLLING OFFICE NAME AND ADORESS 12. REPORT DATE

HQ Air Force Human Resources Laboratory (AFSC) February 1981Brooks Air Force Base. Texas 78235 I. NUMBER OF PAGES

46214. MONITORING AGENCY NAME & AOORESS(II different from Controlling Office) IS. SECURITY CLASS. (of this report)

Operations Training Division UnclassifiedAir Force Human Resources LaboratoryWilliams Air Force Base. Arizona 85224 1S. OECLASSFICATION/OOWNGRADING

IS. DISTRIBUTION STATEMENT (of thl Report)

Approved for public release. distribution unlimited.

17. DISTRIBUTION STATEMENT (of the abstract entered it Block 20. if different frem Report)

IS. SUPPLEMENTARY NOTES

This report consists of two parts. Part I includes pages I through 152. Part Ii contains pages 153 through 460.

15. KEY WORDS (Continue on voer, o side itf neceeary and Identify by block number)

multiple processors data base managementsoftware partitioning software designreal-time computational design evaluation computer systemsflight simulation

M0. ISTRACT (Continue on reverse side It .e..eery and identify by block number)

The overall objective of this study was to design software partitioning techniques that can be used by the AirForce to partition a large flight simulator program for optimal execution on alternative configurations. The resultswere a mathematical model which defines characteristics for an optimal partition and a manually demonstratedpartitioning algorithm design which implements heuristic controls based on the mathematical model statement.

DO tIJAN5 1 Unclassified

SECUFITY CLASSIFICATION Of THIS PAGE (When Date Entered)

AX..

SECURITY CLASSIFICATION OF THIS PAGE(fhon Does Enfet.4)

$51CURITY CLASSIFICATION. OF THIS PAOSurwi Dae RMIw

7- 7.

PREFACE

This report was prepared by the Systems Division, Teledyne BrownEngineering, Huntsville, Alabama. The work vas done under ContractF33615-78-C-0013 with the U.S. Air Force Human Resourcies Laboratory(AFHRL).

Accession For

NTIS GRA&IDTIC TABUnannouncedJustificat io

Distribution/

Availability CodesAvail and/or

Dist Special

TABLE OF CONTENTS

Page

1. INTRODUCTION . ................ . . 7

1.1 Objectives. .. .................... 71.2 Background. .. .................... 71.3 Approacho. ....................... 81.4 Results .. ....................... 9

2. SOFTWARE PARTITIONING .. .................. 10

2.1 Partitioning Environment .. ............... 102.2 Design Goals ....... .............. 142.3 Alternative Approaches .... ............. 15

3. MODEL DEVELOPMENT ................. ..... 18

3.1 Mathematical Statement ................... 183.2 Algorithm.:Design Highlights......... ..... 303.3 Feasibility Demonstration....... .......... 48

4. MODEL IMPLEMENTATION CONSIDERATIONS .. ........... 55

4.1 Flight Training Simulator Evaluation Environment . 554.2 Data Base Management .. ................ 604.3 Target Computer and Source Language Selection ... 714.4 Recommended Implementation Schedule .. ........ 78

5. CONCLUDING REMARKS. .................. ... 86

5.1 Findings. ............... ..... ..... 865.2 Related Work........................865.3 Areas for FurtherSudy............... ... 87

APPENDIX A. USER INPUTS .. ........ ............... 89

APPENDIX B. REPORT FORMATS .. .................... 138

APPENDIX C. FEASIBILITY DEMONSTRATION. .. .............. 153

APPENDIX D. DETAILED DESIGN. .. .................. 365

3plCJN -M M 2AI MalEMW T,

LIST OF ILLUSTRATIONS

Figure Title Page

1 The System Life Cycle ..... ................ . . 11

2 Major Partitioning Algorithm Steps . ......... ... 33

3 User Input Process Flow . . . ........... 35

4 Basic Partitioning Algorithm Control Flow ...... 38

5 Priority Heuristic Selection Process . ........ ... 40

6 Processor Load Balance Meuristic .. .......... . 41

7 Memory Allocation Balance Heuristic .. ......... ... 43

8 Reduce Development Cost Heuristic .. .......... ... 45

9 Augmented Partitioning Algorithm Flow .......... ... 47

10 Report Generator Design ..... ............... ... 49

11 Sample Problem Configuration ... ............ ... 51

12 Sample Configuration Memory ProcessorCommunications ..... .. ................... ... 52

13 Application Flow ...... .................. ... 53

14 Hierarchy of Flight Trainer Documents ......... ... 56

15 Computational Design Evaluation ... ........... ... 59

16 Algorithm Implementation Tasks .. ........... ... 79

17 Projected Time Relationship of Tasks . ........ . 81

4

LIST OF TABLES

Table Title Page

1 Basic Goal Program Matrix Sizing .. .......... ... 31

2 Internal Partitioning Algorithm Control and Look-UpTables Established by PASS I ... ............ ... 36

3 User Input Echo Reports that are Specified inAppendix B ....... ..................... .... 37

4 Development Documents and Their Relationship tothe Partitioning Algorithm for Software Systems . . . 58

5 Data Block Characterization .... ............. ... 62

6 Memory Device Characterization ... ........... ... 63

7 Task Characterization ..... ................ .. 66

8 Processor Characterization .... ............. ... 68

9 External File Sizing Requirements .. .......... ... 73

10 Internal Algorithm Table Sizing Requirements . . . . 76

1-

1t

1. INTRODUCTION

This report documents the Software Partitioning Schemes for the

Advanced Simulation Computer Systems Study performed by Teledyne BrownEngineering (TBE) under Contract No. F33615-78-C-0013 for the Air ForceHuman Resources Laboratory (AFHRL). The report contains five sections.Section 1 introduces the study objectives, background, approach, and re-sults. Section 2 defines the software partitioning problem environment,partitioning goals, and alternative approaches. Section 3 presents thetechnical details of the resultant software partitioning algorithmdeveloped and manually demonstrated under this contract. Section 4addresses implementation considerations and recommends a schedule oftasks for algorithm automation verification and validation. Section 5concludes with a brief recapitulation of the study findings, relatedwork, and areas of further study.

1.1 OBJECTIVES

The overall objective for this study was to design softwarepartitioning techniques that can be used by the Air Force to partition alarge flight simulator program for optimal execution on alternative mul-tiple processor configurations. In particular, the Air Force needs asoftware partitioning algorithm for use in conceptualizing, manipulat-ing, and evaluating candidate flight trainer computational designs.Major design objectives pursued by TBE in deriving the software parti-tioning algorithm included emphasis on potential automated steps, manualfeasibility demonstration, and recomended implementation steps for its

use by the Air Force.

1.2 BACKGROUND

It has been evident for some time that significant increases incomputer system performance may be realized by using two or more smallerprocessors connected in parallel, as opposed to one large processor.This concept has been utilized in many real-time flight simulators whereeach of several computers performs a specific task. Future trends aretoward further expansion of this concept to include not only tasks thatmay be executed in parallel but also tasks that must execute seriallybecause of temporal relationships. This causes many multiple processorconfigurations to be applicable to flight training simulators and com-plicates the problem of allocating the software among the processors.

Typically, the design of a computer system is an iterative pro-cedure. Certain portions of the hardware and software can be designedindependently, but the remaining portions must be designed interac-tively. With the rising cost of software, it has become more and moreimportant to know the effect of computer hardware design on the design of

7

i __

the software as well as the effect of the software design on the selec-tion and interconnection of the hardware to develop the optimum designfor the computer system.

This study has pursued the development of an algorithm that willfacilitate the partitioning of both parallel and sequentially dependenttasks to a given hardware configuration. The algorithm has the potentialof being automated.

1.3 APPROACH

This study was comprised of three phases: Phase I - LiteratureSearch, Phase II - Simulator Analysis, and Phase III - Algorithm Designand Demonstration. This three-phased approach provided a logical

sequence of research and analysis that resulted in the delineation of thepartitioning technique presented in this report.

The Phase I literature search focused on current documentationin two major technical areas. The first area concerned fiight trainingsimulator computational subsystem designs. The second area addressedsoftware partitioning schemes for allocation of parallel and serialapplication tasks to advanced multiple processor configurations.

The Phase II effort was subdivided into two parts. The firstpart was the analysis of literature collected to properly identify thesoftware partitioning goals with respect to flight training simulatordesigns. The second part was the selection and expansion of the specificapproach for the techniques to be applied in the algorithm design toachieve the design goals. Partitioning approaches considered includedmanual allocation schemes, real-time dynamic task allocation schemes,and a mathematical goal program statement of the allocation problem. Themathematical goal program model approach was selected because of itspotential for systematically obtaining optimal partitions and relatedquantitative measures in an automated mode, which are responsive toalternative candidate design features. The features and measures thatcan be modeled are described in Section 3 in terms of the mathematicalmodel, algorithm design, and algorithm feasibility demonstration. Modelmeasures include task sizing and timing; processor utilization; memorystorage, retrieval, and sizing; and real-time task constraints andrelationships.

Some problems were encountered in pursuing the Phase III designto implement the mathematical goal program model when allocating a largenumber of tasks and data blocks to a large number of processors, memo-ries, and peripherals comprising the candidate configuration. It becameevident that a heuristic goal program algorithm needed to be designedthat interfaces with a linear program optimizer to obtain "good" taskpartition allocations for large partitioning problems. TBE's Input/Output Requirements Language (IORL) supplemented with flowcharts was

used to delineate the algorithm design and provide the steps for perform-ing a manual demonstration of the algorithm's feasibility.

1.4 RESULTS

One of the most important results of this study was a mathemati-cal model defining partitioning parameters and measurements. From theseparameters, a set of guidelines has been recommended for the establish-ment of a centralized automated flight training simulator computationaldesign data base repository for the Air Force. These design parametersaddress five major areas, including flight training simulator computa-jtional interface requirements, baseline software task/data descriptions(independent of hardware implementation), candidate hardware configura-tion specification, a technology data base, and (most important) designevaluation user interface data options. These parameters along with thepartitioning mathematical model provide steps for the implementation ofan automated partitioning algorithm for real-time simulators. Detailedrecommendations for algorithm implementation are provided in Section 4.

Section 5 expands TBE's findings, including related aspects ofour Advanced Multiple Processor Configuration study contract encompass-ing areas for further research and development. In the multiple proces-sor area, the impact of heterogeneous processor configurations andpotential reconfiguring capabilities is currently being investigated. Amajor area for future study is the impact of higher order architectureson partitioning allocation.

I I.

2. SOFTWARE PARTITIONING

To develop the software partitioning algorithm design goals, TBE

addressed the definition from both general system software design andparticular flight training simulator software design viewpoints. Thissection supplies the basic definition of the software partitioningenvironment, the design goals selected for flight training simulatorsoftware partitioning features, and alternative approaches consideredduring this study.

2.1 PARTITIONING ENVIRONMENT

To fully appreciate the software partitioning environment andits associated steps, one must first examine its relationship with thesystem life cycle. Then, flight training simulator system life-cyclepeculiarities must be considered. The questions posed by this study inboth these areas concerned the identification of the software applica-tion task features that are peculiar to advanced real-time simulationcomputational systems and that influence the software design partition-ing process. The system and flight trainer life cycles are now described

for the general system, followed by a description of the flight trainingsimulator software partitioning features. Emphasis was placed on iden-tifying software features that characterize an optimal partitioningscheme and that account for alternative candidate configurations andprovide partitions that meet real-time load balance constraints.

2.1.1 System Life Cycle

Figure I depicts the major phases of a system developmenteffort. The development phases that directly relate to or influencesoftware partitioning include subsystem interface requirements, sub-system functional specification, and subsystem detailed design. Inaddition, during the operational maintenance of the system, any changesthat are deemed necessary (to either correct for a design deficiency oroversight, or to implement an expanded capability) imply that a reparti-tioning of tasks may be needed to accommodate the required change. Thisphasing relationship to partitioning holds for any system, whether it isan aircraft, computer center, air defense system, ... , or a flight train-ing simulator system.

For purposes of this study, the detailed design phase wasselected as the major area where software partitioning parameters becomeknown. Prior to this phase, a system partitioning is generally performedto denote the major subsystems and their respective interface functions.After the detailed design phase, actual hardware is procured from whichprototype build implementation is initiated. Therefore, the detaileddesign phase has the greatest influence on mapping software tasks tohardware and vice versa.

10

. . . ,.. a . . . ... ... ... . . .. . . ,,, . ... h " '' d

.. .. '"' "'". ..

Required operational capability

Conceptual ystemsystem development

SystemYerequirements hange

OrequireInterface and

subsystemrequirements Irqimn

Operations andmaintenance

develo Procurent

specification acceptance~~te st ing

, |,

f taDetailed designps o w r d

fverificationBuild, debug, and testing

verification

LW

HW

Figure 1. The system life cycle addresses partitioning at subsystem,function, and detailed design phases for new and/or modi-fied system development efforts.

The design of a multiple computer system traditionally has begunwith the hardware selection. Once the computer system has been selected,the development of software begins. During development and even afterthe system is installed in the field, there are various modifications toboth the hardware and the software. Because software has traditionally

lagged the hardware development activities, the hardware has had adirect influence on software partitioning. As the details of the soft-ware tasks become known, projected hardware resources are typically

found to be inadequate, which necessitates the acquisition of additional

processors and/or memories to meet system interface requirements. Asoftware partitioning algorithm must be able to address software appli-cation design parameters, which are independent of a particular hardwareconfiguration, to permit a variety of design tradeoffs to be evaluatedfor alternative candidates prior to the exact configuration selection.

Once a system enters the operational phase, maintenance becomesthe prime cost factor (indeed, maintenance cost is the largest cost ofthe system life cycle). Change and configuration controls are necessaryfor a system or subsystem of any significant size. As technologyadvances, new software and hardware architectures may need to be imple-mented. A tradeoff must be made to decide whether to convert or totallyredesign existing software. A software partitioning algorithm shouldprovide useful information regarding allocation of current baselinesoftware design tasks to the new or modified hardware architecture. Aswith design development, software partitioning in the operational main-tenance phase addresses the design details of any proposed changes.

The key factor for flexible software partitioning (from the sys-tem life-cycle viewpoint) is the ability to define software designattributes in terms of the dependent application software task/data flowrelationships. The software attributes should remain independent of,but be mappable to, a particular processor architecture. The prolifera-tion of requirements languages (RLs) and higher order languages (HOLs)is a testament to this emerging philosophy in the DOD community. Thedistinction between an RL and an HOL is that RLs are not currentlyautomated to the extent of target machine code generators for the RL. AnHOL such as JOVIAL, HAL-S, or PL-l supports interpretation, data manage-ment, and code generation from machine-independent HOL source code to anintermediate level language that can then be specifically translated toany one of the languages supported by different target machines. Oncethe tasks have been defined in a suitable RL and HOL, the problem stillexists as how they can best be partitioned or allocated to the candidatearchitecture. Once allocated, the resulting partition should be evalu-ated in terms of predicted performance and cost/risk assessments by asoftware partitioning model. Iterative feedback from this performanceevaluation model can then be used to perturb the partition based onperformance penalties to derive a well-balanced software e Xecution

sequence.

12

2.1.2 Flight Trainer Life Cycle

In addition to problems associated with the general system life-cycle environment, the simulation training system environment offersspecial considerations and problems with respect to software partition-ing. Aircraft systems are continuously being upgraded, and this causeschanges to training requirements. manual interfaces change when new ormodified weapons systems, embedded onboard computer systems, and opera-tional tactical policy changes are introduced. These problems arereally no different from problems encountered during the maintenancephase of the actual system. The key issue is when and how actual systemchanges are received, evaluated, and introduced into the trainingrequirements.

Actual system test and performance measurement tools can andshould provide useful inputs for simulator training software required tosupport the new/modified devices. In the case of embedded computer sys-tems, simulated training scenarios could provide additional reliabilitytests of the actual onboard computer systems as well as the prime goal oftraining personnel. As a result of these considerations, the partition-ing algorithm should facilitate modular design definition input changesand permit new technology configurations to be introduced as needed tosupport a given evaluation. This should also include the ability to fixallocations of certain functional tasks, such as a set of onboard com-puter tasks, while permitting others to be allocated by the partitioningalgorithm.

AFHRL supplied a benchmark problem and the detailed design docu-graduate Pilot Training (ASUPT, now known as Advanced Simulator for

Pilot Training (ASPT)). These documents were analyzed to obtain esti-mates on the complexity and sizing of flight simulator software parti-tioning. This analysis identified 50 major application (both real-timeand support) tasks (some of which would be duplicated to support multipletraining stations, instructor consoles, weapon systems, and aircraftmodels). The results of this analysis were presented at an interimbriefing.

It should be noted that a task is related to the application.Its ultimate operational realization may be software, firmware, hard-ware, or a combination of these, depending on the selected design config-uration. The tasks being considered for the partitioning algorithm arerelated to the computational subsystem of real-tine flight training sim-ulators.

Further analysis revealed that the trainer computational sub-system is really comprised of a set of smaller functional subsystems,such as simulator facility control, visual computational support, andsimulated aircraft mathematical models; thus, the number of processorsand number of tasks for which selected software functions are being

13

1I

allocated is reduced to approximately 30 tasks to three processors usinga comon, shared multiport memory. In summary, flight trainer computational configurations have both a functional partitioning of processorsand a task partitioning within each functional processor group.

2.2 DESIGN GOALS

Software partitioning of tasks to alternate candidate multiproc-essor configurations must be a systematic process based on measurableevaluation goals. The selected design goals for the partitioning algo-rithm developed are as follows:

(a) With software system task flow inputs given, partitiontasks to a user-specified multiprocessor hardware configu-ration subject to input constraints

(b) Identify interdependencies among the tasks that requirecomiunication links

(c) Incorporate dynamic performance evaluation feedback todetermine the best partition to preclude system deadlocksand account for critical path task precedence orderings

(d) Provide a means of balancing the processing load as a func-tion of processor utilization, which is evenly distributedamong the processors such that no one processor is satu-rated while others remain idle for appreciable periods oftime

(e) Provide cross reference of task(s) assigned to each proces-sor and processor(s) assigned to each task

(f) List critical constraints when a valid partition is notobtainable

(g) Provide a development cost estimate as a function of tasksizing and instruction mix, which is related in terms ofassigned candidate processor language compilers and debugtool measures.

In deriving this set of goals, several issues have been dis-cussed pertaining to the evaluation environment in which the partition-ing algorithm is to operate. The baseline set of questions was:

(a) At what point(s) in the system development cycle is thealgorithm to be used?

(b) What timeframe and computer resources are anticipated forcandidate evaluations?

14

- 'a, i . . . ..-

(c) To what extent will the system requirements be formatted?In what format?

(d) To what extent will the alternative candidate design con-figuration be documented? In what format?

The answers to these questions relate directly to the level ofsoftware partitioning and types of system parameters that can bemodeled, allocated, and measured. In summary, there are no definitiveanswers to these questions since each flight trainer evaluation tends tobe tailored to specific needs. This does not mean that systematicmethodologies and standards do not exist, but they do differ from oneproject to another. The potential use of an automated partitioningalgorithm will require systematic collection and development of flighttrainer requirements, software specifications, and candidate configura-tion inputs. This contract has concentrated on the definition of parti-tioning algorithm logic in terms of design inputs which are transformedvia technology data and user evaluation options to assist and assess thepartitioning of tasks for a given candidate configuration.

2.3 ALTERNATIVE APPROACHES

Software partitioning to date has been primarily a manual proc-ess based on experience gained in development of previous flight simula-tors. The designer comunity continually evolves and improves partitionallocations using projected resource requirements and implementing thepartition to see how well it performs. In some cases, real-time alloca-tion is determined by a master computer using a predefined assignmentscheme that incorporates certain dynamic application considerations.These schemes, whether manual or partially preprogramed controlled, arenot easily automated, since they generally require that a specific sys-tem allocation be implemented for a given configuration. Manual projec-tions are limited to a few alternatives for a given type of configura-tion, but they must be redone for alternative configurations.

In surveying potential automated models to meet the designgoals, the basic problem to be solved is one of distributing the softwaresystem tasks and related data blocks to a candidate hardware architec-ture network such that a representative stressing simulation load ishandled. In general, this type of problem is typical of mathematicalprograming problems addressed in an operations research (OR) environ-ment. Within this field, there are a variety of algorithms. The follow-ing are some of the more familiar:

1. Transportation problem of product transport from productionlocations to warehouses and customer distribution centers tomeet customer demand at minimum cost.

15I. S

* .

2. Traveling salesman optimal route determination to servicecustomers

3. Knapsack packing of items required for a camping trip to bedistributed evenly among campers

4. Capital budgeting problem of choosing among independentinvestment alternatives to maximize return subject to cur-rent investment fund constraints

5. Machine shop production scheduling to meet product demanddeadlines with minimum machine restructure between jobs andgiven employee mix.

The software partitioning problem has attributes similar to each ofthese.

In the case of the software partitioning problem, a descriptivestatement of the model is as follows:

1. Find a partition that best satisfies alternative evaluationpriority functions:

a. Balance the processing load among the processorsb. Balance the memory storage utilizationc. Minimize development costs.

2. Subject to:

a. Real-time task resource requirementsb. Predicted performance simulation feedback.

When defining a software task partitioning model, a number offactors must be considered. The model can very quickly get out of handin terms of size for current optimization techniques. Thus, the modeldesign developed under this contract restricted itself to a static allo-cation problem that is mathematically stated as a linear goal programproblem in Section 3.1. It is static in that it is a generalization ofthe real-time application tasks to be allocated to a given candidateconfiguration. In this sense it is not a dynamic real-time allocationalgorithm. The static model is very useful in the candidate designevaluation mode, since many numbers are based on predicted task sizingand timing plus anticipated computation iteration frequencies to supportgiven training loads. The static model permits average to worst-casegrowth analysis in a systematically controlled evaluation environment,which provides the means to ensure a complete design description has beeninput and independently provides a measure of processor utilization,memory utilization and predicted software development cost.

16

Even in the static model environment, optimization data basesizing and numerical roundoff problems are encountered for evaluation ofa computational system involving much more than three processors, 20tasks, 40 data blocks, and four memories. Specific sizing is addressedin Section 3.2. For this reason, a heuristic model has been designed. Aheuristic model is a means of limiting computations to a logical sequenceof iterative improvements via allocation tradeoffs until a certainobjective level is either found to be feasible or a bottleneck has beenisolated.

This section has discussed partitioning considerations. The re-sultant algorithm design details are highlighted in Section 3. Imple-mentation considerations are given in Section 4. Section 5 incorporatesareas for further research with respect to optimizer techniques and database selection.

17

3. MODEL DEVELOPMENT

Software partitioning model development is presented from threedifferent technical viewpoints in this section, including the mathemati-cal definition, the detailed design highlights, and a feasibility demon-stration synopsis. The model is expressed in generic computationalsystem terms where the major components are tasks, data blocks, proces-sors, and memories that are partitioned to service an external baselineload environment. The mathematical model definition delineates all theparameters and the basic relationships that must be satisfied for a validpartition. It also provides a statement of objective functions thatpermits optimization of the partition when the basic relationships arefound to have a feasible solution (i.e., a feasible partition).

The algorithm design highlights are presented here in terms ofthe systematic procedural step features with cross-references todetailed appendices. Appendix A provides user input information. Out-put report formats are provided in Appendix B. Appendix C contains thefeasibility demonstration that emphasizes the user environment of inputformulation, critical intermediate step results, and final output summa-ries. Detailed computations and design logic are enumerated inAppendix D.

3.1 MATHEMATICAL STATE1ENT

This mathematical statement provides mathematical terminologyand definitions for alternate evaluation priorities and constraint form-ulation based on a generic statement of a candidate configuration forwhich a set of software tasks are to be partitioned. Each mathematicalsymbol is defined when first introduced. In addition, Appendix C con-tains a master list of mathematical symbols and related design defini-tions. A special effort has been made to use a unique symbol for a givenentity. It utilizes a combination of symbol definition with a combina-tion of linear programing and goal programing model formulation termi-nology. Although knowledge of thepei modeling and solution techniques ishelpful, it is not essential to the understanding of the basic expressionof the software partitioning problem model.' The solution techniqueswith respect to the software partitioning model are considered in thedesign highlights of Section 3.2. The model is now stated.

3.1.1 Mathematical Terminology

The mathematical model formulation permits the major decisionvariables to be enumerated in terms of a baseline software load for a

1Ignizio, James P., Goal Programing and Extensions, D. C. Heath and Com-pany, Lexington, Massachusetts, 1976

L

given real-time interval of length, T. In the case of the flighttrainer, T might be chosen to represent the maximum tim permissible fora complete real-time cycle. The baseline load could represent astressing training mix of tasks and data relationships that must beperformed to support the given trainer facility exercise; for example, atwo-on-one, air-to-air, combat maneuvering situation may be selected.For more detailed partitioning loads, T could be selected to represent aspecific segment of the real-time cycle to further analyze and partitionparallel versus dependent task/data flow relationships.

The major decision variables (outputs of the algorithm) with re-spect to software partitioning allocation are defined as follows:

xtp , if task t is assigned to execute on processor p

= 0, otherwise

e -p number of task t executions on processor p for theevaluation problem time period

ytp development cost to implement task t on processor p astp currently partitioned

Smb - I, if memory storage m contains block b

W 0, otherwise

hb - number of memories where block b is stored

ampti - number of times input block i of task t is input fortask t on processor p from memory m.

wmpto - number of times output block o of task t is written orupdated by tudk t on processor p to memory m.

These outputs are determined for a given set of software task and candi-date architecture inputs. The basic algorithm control inputs aredenoted by:

T - number of tasks to be allocated to processors

P - number of processors

M - number of memories

B - number of distinct storage blocks to be allocated to memories(this includes instruction and data blocks)

.1 19

Q - number of communication links

8 - maximum number of input and/or output blocks per task.

The values of these parameters control the overall algorithm sizing,timing, and looping logic.

The baseline task load may be represented as configuration-independent, processor-dependent, and memory-dependent input parame-ters. The configuration-independent input parameters are defined asfollows for each task, t:

N - number of times task t is to be executed during the evalua-tion interval, 7, for which partitioning is being done

St - maximum time limit per task t execution

I - number of distinct input blocks for task t

Lti - global data block index for task t input block i

Ati - percent of information input for task t from block i

0 - number of distinct output data blocks for task t

0to - global data block index for task t output block o

0to - percent of information output from task t to block o.

The processor-dependent task inputs are defined as follows fortask t on processor p:

c - time for task t execution on processor p

R - resource task management coefficient for task t on proces-Rtp sor p if time or data enabled task (these tasks require

periodic enablement or polling by the processor to whichthey are assigned)

rt- resource task management per task t execution on processorrtp p for slaved enabled task (these tasks are enabled by

another task)

dp- the cost coefficient for developing task t to run on proc-tp essor p independent of allocation

6tp the cost coefficient for resource management of task tdevelopment on processor p.

20

Section 4.2 discusses the implementation means for computing thesevalues based on independent task descriptions, processor configuration,and a technology data base. The mathematical model assumes that thesevalues are known.

In addition to the task-to-processor allocation relationships,the storage allocation of blocks to memories operates on a similar con-cept. A master block list of distinct data and/or instruction blocks isindependently defined and then mapped via the candidate configurationand technology memory parameter inputs to supply the following parame-ters with regard to block b, memory m, processor p, and communicationlink q:

t - length in bits of block b when stored in memory mmbL - length of memory m in bits

a - 1, if access from memory m to processor p exists, i.e.,mp there is at least one access link q for m and p

0 0, if otherwise

a- bits/second transfer rate from memory m to processor pmp based on statistical composite of access links for p and m

W = 1, if processor p is permitted to change contents ofmp memory m, i.e., there is at least one write access link q

from p to m

-0, if otherwise

W- bits/second transfer rate from processor p to memory mmp based on statistical composite of write access links for p

and m.

The task relationships to these blocks are defined as part of the real-time constraints in Section 3.1.5.

3.1.2 Processor Utilization and Growth Balance

Given the mathematical terms defined in Section 3.1.1, the proc-essor utilization, U , associated" with a partition may be expressed asfollows (for each processor p-1 to P):

T

u =; I (c +r e a task computationt tp ~I'~and resource

management time

21

t M

_ -1tid %mtia uti task inputi-i1 u p processing

v t task output0t QtO

o1 1 Omp~to mpto processing

+ R task resource+ tp Xtppmanagement

An absolute constraint is that-

Up 4 1 for p-l to P.

In other words a processor, pp cannot be more than 100% allocated.

The objective function for processor balance may be written:

P-1 PMinimize 1 EUi - TUj . Minimize differences

i-l j-i+l in processor loads

It should be noted that the presence of absolute values implies a non-linear objective. The processor utilization balance can be mapped (via aranked ordering of the U * U' such that U.' > U.') to a linear objectivefor a given partition. k

This objective statement assumes that perfect balance is theultimate or optimal partition. The candidate design being consideredmay represent only a portion of a bigger design evaluation problem. Inthis case, the use of certain processors way be favored, whereas othersshould not be considered. To handle this more realistic partitioningsituation, each processor has two additional parameters, which are user-specified:

absolute upper limit for processor p's utilization

22-[ 4

Gp - goal or target limit for processor p's utilization.

With these additional parameters, the following constraints apply:

Gp L p Goal must be

less orequal to theabsolute limit.

U Each processormust be belowits absolutelimit.

The objective for the optimal partition in terms of processor utiliza-tion becomes:

P-1

Minimize a (Ui-G i) - (Uj-G)

i-l j-i+l

This basically states that the processor utilization is in balance withrespect to user-specified goals. In the case of a flight trainer soft-ware partitioning evaluation, G could reflect a percentage that allowsfor future growth. Thus, 0 P0 . 60 reflects a 40Z growth factor forprocessor p.

The algorithm as currently designed (Section 3.2) assumes thatan initial feasible solution is provided by the candidate design andutilizes a heuristic solution based on the absolute difference betweenthe most heavily loaded processor and the least loaded, taking intoaccount the goal growth reservation to distribute the process load.

3.1.3 Storage Utilization and Growth Balance

Storage utilization, u m may be expressed for each memory unit,m-l to M, as:

u a - abb. Sum of blocksU b-l stored divided

by total memory

23

As with the processor balance formulation, storage utilization cannot

exceed the capacity of the device.

um :1 for m1l to M

In addition, storage growth balance can be established with arespective goal utilization, gm , and an absolute limit, 61., for eachmemory as follows:

M-1 M

Minimize (u - ) - (u.-gj)i-l j=i+l 1ui-gi

where

u : for m=l to M.

As with the processor utilization, the solution techniquedefined in Section 3.2 for storage utilization is based on a heuristicdriven by the most used and least used memory allocations with respect toinput goals.

3.1.4 Development Cost

Software development costs are a function of task complexity andprogramming support tools available. In particular, the heterogeneousmultiprocessor system adds another development cost concern, i.e.,coding of a task to perform on more than one processor type. A commonprogram source language significantly reduces duplicated coding efforts.Thus, the development cost for a given software task, t, in the model maybe stated as:

Djt E d t p x t p p one-time development

p=1I

+ 6 tp resource manager development

- dtp Ytp duplicate utilization.

where

ytp 0 for p-l

ipt t

24 p~p.

where -kip t i 1, if an identical source language isavailable on processor i and p (i 0 p) for

task t

= a technology-specified constant if differ-ent languages are to be used (i Op)

= 0, if i f p.

If the code already exists, then dtpfO.

Note that the multiplicative factor for determining y can be stated asan equivalent series of linear constraints because of e zero-one vari-able x (task t is either assigned to processor p or it is not). These(p-l) Anstraints are enumerated as follows for a given task t on proces-sor p (for p > 1).

Ulpt xti - Ytp 0

\2pt xt2 -Ytp 5 0

X(p-1)pt x t(p-1) -Ytp 5 0.

With this set of constraints, minimizing yt in the achievement functionensures that y will assume the appropriaie maximum as defined in theoriginal defind'ion.

The goal objective for software development cost is now statedas:

T

Minimize E Dt-t=l

This is basically a problem of reducing development cost. The designattempted to reduce development cost (Section 3.2) to be less than a usersupplied value, D, where D represents a ballpark estimate for the totalsoftware development. The unit used may be man-years or dollars, depend-ing on units established for the technology data base (described inSection 4.2), which will be used to translate the task t instruction mix(Section 4.2) to its one-time development cost (d tp) for processor p.

25

.f

The comon language coefficient, X. , is also a function of the tech-nology-processor-related data (SecWton 4.2) and the language factorselected for the task.

3.1.5 Real-Time Task Resource Requirements

The major constraint areas interact with the objective priorityevaluations to further specify acceptable partitioning attributes. As aminimum, the following constraints apply to basic task resource require-ments and processor accountability:

(a) Each task, t, must be assigned to at least one processor.This implies T constraints of the following:

E=1 X tp 1 for t=l to T.

(b) If more than one processor is permitted to perform the sametask, a resource management overhead will be allocated totask t processors via the processor utilization objectiveof Section 3.1.2. However, to ensure that x is properlycoupled with etp, the following constraint musbe applied:

e tP 0.1 tp Nt

In addition, constraints must address task iteration rate and task ser-vice times to ensure that real-time task timing requirements are met:

(a) Given that task t must be executed Nt times during theproblem time period, 7, the task iteration rate constraintis:

p

p=l etp Nt.

(b) If overlap of task t execution is not permitted (i.e., tcannot be executing on more than one processor at a time),the following constraint applies:

E (Ctp + rtp e < minimum (T, Nt *St)

pal

26

A *.. -

where St is the maximum time limit for one execution of taskt.

Note that if

Ctp + rtp > t

then e can be automatically assigned a zero value anddeletelPfrom consideration.

Task data dependencies must also be satisfied. These constraintsinclude:

(a) All data blocks associated with task input must be availa-ble to the processor(s) that are permitted to perform thetask. Thus, for input block £til the following holds:

M+ E a s, 0-X1p mp m~i

for il to It, tl to T, p-l to P.

(b) All data blocks associated with task t's output must residein memory storage m, which can be updated (changed) by anyof task t's processor(s) p. If x p satisfies

tp Xtp

then for a given task output, block b=o 0 the followingholds:

Ht+ xtP + E Wmp Smb -hb >I

m=1

for t-l to T, o-l to O, pal to P. hh represents the numberof different memories that have dupicate copies of blockb; thus, this constraint requires all duplicate blocks tobe updated (see next constraint set).

(c) Any duplicate data blocks must be held to a minimum; there-fore hb may be thought of as a penalty to be added as anadditional objective function with the following additionalconstraint:

27

OWN--

hb > 1 (at least 1 block is in memory)

and

M

, mb - hb - 0

m-1

for b - 1 to B.

(d) Input timing must properly account for the number of task texecutions on processor p (e tp) for each task input block,ti' i-l to I t

Metp - E ampti - 0

mptlrn-1

and ap s mpti 0 for m-l to Mti Nt

are used to ensure that ti is available on memory m.

(e) Output timing must account for the number of task t execu-tions on processor p (e tp) for each task output block, 0to,o=I tO t:

et -V -N (0- ) 0tp mpto t ( Mto

and

- mpto 0 for m-i to MWmp mOto Nt

are used with a corresponding achievement function thatminimizes w to ensure that all duplicate blocks of 0oare updated p t to

28

,...........

3.1.6 Performance Simulation Feedback

Sections 3.1.2 through 3.1.5 comprise the fundamental modelobjectives and constraints that must be set in terms of a valid staticallocation of tasks. Performance bottlenecks detected by the simulationmode being developed under separate contract (No. F33615-79-C-0003)will add additional constraints and/or modify coefficients. In particu-lar, the data transfer objective coefficients for given interfacesbetween a memory and a processor may be readjusted to penalize use ofcertain processors for a given task and/or memories for certain datablock allocations.

A stronger set of timing constraints may be required for depend-ent software task threads. A task thread., Fk, may be defined as a groupof serially dependent tasks with the following notation:

Fk " fkl' '" kG

where f indexes one of the T tasks. In general, task fk9 must haveexecute&fkV percent before task F can be enabled. Thus the tasksdefined as a thread are not permitt&ko run simultaneously in parallelprocessors. This constraint may be written for each thread k as follows:

Pt tpk (ctp + rtp) etp + Rtp xtp) minimum {TTkl ,i

tZF k p=1

for k-l to K, and Tk represents feedback timing for thread K. A furtherassumption is that if task t is an element of a software thread, Fk, thentask t may not be an independent task or an element of another taskthread. If a task is required in more than one way, it can be defined asa group of different tasks for partitioning purposes.

In general, these threads represent critical system task pathflow bottlenecks as determined by the performance simulation of a givenpartition allocation. The algorithm introduces new or revised con-straints until one of the following conditions exists:

(a) Satisfactory solution found(b) Infeasible condition identified(c) Maximum feedback iterations performed.

The current solution state is to be saved and/or printed for futureevaluation as requested by the user evaluator.

29

3.2 ALGORITHM DESIGN HIGHLIGHTS

There are many mathematical program techniques, including bothlinear and nonlinear optimizers and heuristics. The partitioning modelrequires integer solution values that immediately classify it as a non-linear global optimization problem even though the model itself consistsof linearly expressed objectives and constraints. In addition, two ofthe three achievement priority functions (i.e., balance the processorload and balance memory storage) are nonlinear in their formulation ofminimizing the sums of absolute differences. These nonlinear goalscombined with the goal program matrix, which is sized according to theparameters represented in Table I, would be a challenge to both sizingand timing of comercially available mixed integer linear program modelswith a single achievement priority.

To determine the viable design alternatives, a study of goal

programming was made, including several military goal program applica-tions that have been implemented. Applications included weapon systemslice optimization in relation to planning force analysis and a balancedbudget allocation model for mixed project/agency funding. Both of theseapplications interface goal programing models with other analysis tools(such as simulationg input/output analysis, and regression analysis) toprovide a set of automated operational evaluation tools. Theseadditional tools provide a means to cross-check and supply detailedmodel data values that are used to calibrate the goal program model. Thecalibrated model is then used for selected parametric studies todetermine impact on solutions in terms of parametric margins andsolution sensitivities. Both of lese applications utilize modifiedversions of the classical textbook '

- multiphase goal program computeralgorithms. A major drawback to these codes is their susceptibility tonumeric roundoff error propagation for problems involving more than 50to several hundred variables and constraints. In addition to thenumerical roundoff errors, the multiphase codes studied do not usedynamic core memory management. This requires the entire matrix andassociated bookkeeping variables reside in main memory.

In lieu of funding the development for a mixed integer goalprogram optimizer for larger problems, an alternative algorithm is thesequential use of a good commercially available linear program optimizerinterfaced via a goal program driver that introduces each achievementone at a time. This permits continuous solution problems with up to16,000 rows to be handled, given adequate dynamic disc storage. Currentstate-of-the-art integer solutions are restricted to several hundred

Ignizio, James P., Goal Programming and Extensions, D. C. Heath and

Company, Lexington, Massachusetts, 19762 Lee, Sang M., Goal Programing for Decision Analysis, Auerbach Pub-

lishers, Philadelphia, Pennsylvania, 1972

30

a a l a a to

C) -C;

rD Ch c -4 -

-LuJ LuI

cr C'J~ C;4 + 0)

S+

CFE + .

C)+ +

C-' I-i C'* C'J 0I c

w + + -

0-4 Z: \J

a- + + +

LLI - -.

___ ___ ___ ___ ___ ___ + +

en m i if I

0

0C)

LU (J m rA .0 0

31

.'r. .A I&

variables. The sequential use of a linear program optimizer is theapproach recommended for further study in addressing a subset of thesoftware partitioning algorithm as designed in this study. The designhas remained independent of a specific computer optimizer code.

Even with the sequential mixed integer linear program technique,the sizing of the partitioning problem (given in Table 1) is prone tochallenge the best optimizers without some careful matrix selection gen-eration techniques. There are two major areas of concern:

1. The time consumed in determination of an initial feasiblesolution

2. Excessive iteration thrashing to determine "optimal" integersolutions.

The study of goal programming included a survey of heuristic techniquesthat can facilitate the search for improved solutions given an initialfeasible solution. In practice, application-customed heuristic algo-rithms have provided an efficient means for handling and reducing thelarge solution space of alternatives to be searched.

1

In the case of flight trainer candidate designs, the designershave an implied partition which can be used as the initial solution. Thepartitioning problem then becomes one of "Does a better solution existwith respect to load balance, memory balance, and development cost?" Theincorporation of an initial solution step has been recommended as animplementation step requiring further study for obtaining an expandedevaluation capability. The current algorithm design assumes that aninitial solution is supplied and proceeds in a heuristic manner to seek abetter solution.

To achieve a well-defined user evaluator interface of partition-ing input data, a customed heuristic goal program driver, and solutionsummary capabilities, the Partitioning Algorithm for Software Systems(PASS) has been designed emphasizing the four major processes denoted inFigure 2:

1. User input interface and processing referenced as PASSI

2. Basic partitioning algorithm referenced as PASS2

3. Augmented partitioning algorithm (PASS3) to handle dynamicperformance prediction feedback

Ignizio, James P., "Solving Large Scale Problems: A Venture into a New

Dimension," Pennsylvania State University, 1978

32

.' c c

+ - U,~o.Sc " 40c

a c :0l 1. I + "

• ... - • .I L ...-.

inb

0-,

u o-1 ! •. ~l4t1

16,l I

SO 6 v C "6 C 3 -

9 .0 9 C 0 0 * a

009,J <

I I-

<0

33 I .9

... 3~ m I j• * " t

+ 1/-. 011)

h - La IUI .O41 ioou a

~ 41a.33~0-1 *-141 SC

* 341*..5AA ,w,,CU,.+- .--a 000* 4 1 3 h a 0541+,..- '~ ' I-- -+.+.t,.

4. Solution summary reports (PASS4) of a given partition forcandidate design i.

Prior to describing each of these steps, the overall design flow of thesteps and their interfaces is presented.

The major external interface (exclusive of an optimizer) withPASS include the evaluation user and a multiprocessor configuration per-formance predictor simulator. The user interface considerations foractual implementation are expanded in Section 4, with emphasis onincorporating a modular, automated data repository to facilitate inputpreparation of PASS1 and maintenance of current flight trainer designparameters with respect to given partitions (PASS4). The performancepredictor interface is designed to interact with the Computational Per-formance Predictor Simulator (CPPS) being specified and designed underseparate contract. The iterative process of determining a new alloca-tion (PASS3) based on performance prediction feedback is performed untilone of the following conditions is reached: (a) satisfactory partitionis found, b) design bottleneck is identified, (c) maximum iterationshave been reached.

3.2.1 Input Processing Step PASS1

The mathematical statement of Section 3.1 contains software,hardware, and combined software/hardware parameters. The design effortsof this study have emphasized the separation of any combined parametersinto basic hardware and software components with the aid of technologydata base tables and computational formulas necessary to generate thegiven "combined" parameter. Thus, all task/processor and data/memoryparameters are derived from independent software and hardware designconfiguration inputs (see Section 4.2).

The specific inputs are defined in Appendix A. Figure 3 deline-ates the major design process flow for user input editing and computa-tional sequences to properly set up for the actual partitioning stepsthat follow. The design demonstration (Appendix C) provides thedetailed computations to map the user input into the internal partition-ing algorithm control and lookup tables listed in Table 2. Appendix Bprovides representative report formats for the user input echo, whichconsists of the reports listed in Table 3.

3.2.2 Basic Partitioning Algorithm (PASS2)

This step provides the basic controls and logic for interfacingwith the three user-ordered heuristics to determine whether an improvedpartitioning solution can be found. As mentioned in the introductoryremarks on design in Section 3.2, the basic assumption is that an initialfeasible (with respect to real-time constraints) partition is supplied.The resultant basic partitioning algorithm flow is denoted in Figure 4 as

34

at4 O x Z S

4. 0 ; wo S!

.1 >I Z X h.U U

0 0.

Z j.

CS4 toIMI:In MAM Uj l-

114 UinzO! * 2".WI4

1JM L L W UIN ..

ve we4 .1 w, - 40 0 0

IL L ~U U 0wi 0S

- -I SI4--

-43 owl___1- a. Z 19

It 1- .4 ;- w 0 w j

TABLE 2. INTERNAL PARTITIONING ALGORITHM CONTROL AND LOOK-UPTABLES ESTABLISHED BY PASS 1

GRP TABLE TITLE

1 Limits, Constants, and Codes

2 Current Problem Sizing Controls

3 Priority Controls

4 Current Processor List

5 Current Memory List

6 Current Communication Link List

7 Current Internal Device List

8 Task/Processor Allocation and Restrictions

9 Memory/Processor Allocation and Restrictions

10 Block/Memory Allocation, Restrictions, and

Coefficients

11 Master Block List

12 Master Task List

36

TABLE 3. USER INPUT ECHO REPORTS THAT ARE SPECIFIED IN APPENDIX B

FORMAT* REPORT TITLE

1 Standard Run Identification

2 Hardware Component Summary

3 Data Block Summary

4 Task Summary

5 Baseline Load Summary

6 Evaluation Options/Restrictions

7 Evaluation Priorities

8 Basic Partitioning Problem Size

* Format reference to Appendix B

I

161 1:1Nt

1mh..I V2;

- z 41

1. 0 of wC

I2 ) 49I -co

ki Im 0 0Wo w s-

hi U 40 EU

I- ~ ~ $ A~i2W 4

ai N 0.10

t% 4 ~ mU fL.'A.I-JII-hi L 4 a a

Lw2>ZZ> 38

IMbeing comprised of initial solution verification, heuristic controltable setup, and user-specified, priority-ordered heuristic executions.

There are three basic heuristic algorithms corresponding to thethree objectives or achievement functions: processor utilization(LOADBL), memory utilization (MEMBAL), and development cost (RDCOST).Figure 5 denotes the major selection branch as being a function of theuser-specified priority execution order GOAL (g), where g is the currentpriority level being executed. Prior to invoking the appropriate heu-ristic, a test is made to determine whether the basic priority goal levelhas already been achieved. If so, a return is made to proceed to thenext priority level.

The major features incorporated. in the design permit ranking ofthe current partition solution variables with respect to impact on thegiven priority under consideration. The following ranking definitionsare utilized for each of the respective heuristics:

1. For the load balance heuristic, processor p's utilization,U , is subtracted from its goal, G , to define U' - G - U .T~e resultant U' array is then ranted from high tS lowPvalugs(i.e., those below their goal to those above their respec-tive goal in order of difference magnitude). The resultantranked array is then used to determine whether the load iscurrently in balance, i.e., (U'1 - U' GTOLPU) with respectto a user-supplied tolerance (4TbLPU) for processor utiliza-tion. The object is to offload some of the tasks from theheavily utilized processors to the lighter loaded processorsto obtain a better balance, as denoted in Figure 6.

2. For the memory balance heuristic, the allocated memory, um,is subtracted from its goal allocation, g , to define u' m=Im -um . The resultant array, u' , is Then ranked (in asimilar fashion as processor uthlization) to determinewhether the current memory allocation is in balance accord-ing to the user-supplied goal (u' -u' GTOLMU). Theobjective (Figure 7) is to reallocate Msome of the blocksfrom the over-allocated memories to the under-allocatedmemories to obtain a better balance.

3. The development cost is a minimization problem of individualtask development cost. Thus, the tasks are ranked from mostexpensive to least expensive. The ranked cost array canthen be systematically processed (Figure 8) to determinewhether a more cost-effective solution is possible (i.e.,can this task be implemented on another processor in thecandidate configuration of less development expense andstill meet real-time constraints?). It should be noted thatthis priority is only applicable to a heterogeneous set ofcandidate processors.

39 J* .°

4

b, u00

- hJI I- zWa <

0 uU 0

N U;i

z 0

zIO o Sin hi

01 1-- .II-0u u

I .- I 1~.-gI- ~~r~* _____J_

A-I >4U

UZI.

pm u -C - 'j. 4-

JU !5-

*, L..4.~ OON

). o=

40

ilt D49LII

J ~ ~ ~ ~ I AGUAAIOLICSLIS'.40=990111

WHICH_2_ ARC .|I

ON IL644AL Pm EiO

__i . J- ASS aaS~.I LIN r

J l I F -NP erf"9- -1 A--srAP e o SACLOCATiIO...

-4 " I" j?

iFOCC so:*~ a

A4L1[ IT*IL

pGGALSL VEL

CAN La.; I.

SamO~ 0,0 '

60 'IA IjJlJ!IJLIf

opt 1-61Imm GOV L i A WNSWESgU

MALLE ALUCA 16

Figure 6. Processor load balance heuristic.41

a ..-... . S

oI~c!PieN~Pik

4 4 iU

AlN

a -q

ZL 0-

Ito-

'liii 42_ _ _ _

~~*~---~*t PA6W 6ALUCAV 166

pn ge CAC44 Ja e I66 ADS"S LUTC LON1

1"" au IL :OVALO"1 1

CI.

In f ew #,*"a y

we or I C*.flAI: SA--.- -6*..* 950W1

PjA i. a,IV"AIpa~euuaeiu 0IAj

UIMletW I

Figure 7. Memory allocation balance heuristic.43

j~

-;; .0

44a

i 0 ofI~0 bI (SO A

z -a i0a ly 2

4N **'4~ coo

0 b,

J 4

U, z j

a n,

fe IN).9

N IL I* ..

-. .. .-, >

For each of the heuristics, checks are incorporated to ensurethat real-time limitations are not violated by any subsequent new"improved" solutions found by the respective heuristic. Design emphasiswas placed on the order for incorporating these checks within the heu-ristic procedure to avoid excessive calculations when easily determinedrestrictions would prohibit exploring a given tradeoff. For example,when attempting to reallocate a task to another processor, only thoseprocessors that may perform the task are considered. To solve some ofthe more complex interrelated real-time constraints, a linear programstatement might be studied to determine whether effective utilization ofan optimizer would be feasible for performing the given tradeoff. Thecurrent algorithm incorporates a specific check of constraints as formu-lated in Sections 3.1.5 and 3.1.6.

The heuristic driver continues at each priority level until ithas exhausted its systematic exchange tradeoff search for an improvedmeasurement. The three priority levels are executed in the order asspecified by the user evaluation priority inputs of PASS1. The basiccomputational and logical sequence flows for each of the three prioritylevels are denoted in Figures 6, 7, and 8, respectively.

3.2.3 Augmented Partitioning Algorithm (PASS3)

This step is an expansion of the PASS2 processes with emphasis onresolving identified performance bottlenecks of the following types:

1. Cycle or thread timing is not sufficient for real-timesystem response.

2. Specific candidate component (i.e., processor, memory, com-munication link) utilization is unacceptable.

The basic process decision flow is depicted in Figure 9.

Recognizing that manual user evaluation insight may help expe-dite the search for an improved partitioning, process PA 3100 facili-tates the option that the current allocation can be manually modified.Once any modifications have been processed, the performance data areprocessed via PA 3200 to readjust coefficients and to set up additionalconstraint generation controls. The new constraints are then con-structed and their basic impact on the current partition is assessed interms of solution feasibility. Each performance bottleneck is processedindividually, in a predetermined order of criticality during this pro-cess (PA 3300).

If a cycle or thread is the bottleneck, then the respective re-source management and data communication links are examined to determinethe major bottleneck within the thread or cycle. Penalty coefficient

46

I-I,

2w I- I o3

m0 -I

b-h '4

4m 42

Wi 4 u

40,

U. '3

'3 0

adjustments are made to the processor utilization equation. An alter-nate partition is sought that satisfies the end-to-end time requirementof the given cycle or thread under these more stringent constraints.

If a component is above its allotted utilization, a check as toprocessor or memory balance bottleneck is made. If it is a processor,the processor heuristic is used to offload the offending processor. Ifit is a memory problem, an attempt is made to find a faster access memoryor add a duplicate block if shared memory access is the bottleneck.

As the processing of bottlenecks is performed, the augmentedheuristic driver invokes PASS2 partitioning modules interspersed withadditional checks for maintaining the appropriate thread and/or cycleconstraints. If a new partition is found to be acceptable, it is savedfor feedback to the performance simulation and further manual analysis.If not, the problems are identified for user evaluation. Appendix Dcontains the detailed design flows necessary to fully enumerate thealgorithm. Additional changes are anticipated as the details of theperformance simulator design are enumerated under Contract No. F33615-79-C-0003.

3.2.4 Solution Summary Reports (PASS4)

The report generation features of PASS4 are designed to provideprinted summaries of a partition found by either PASS2 or PASS3 for agiven candidate configuration. The specific formats chosen present thepartition solution from five complementary, but different, aspects,including (a) partitioning priority level measurements, (b) task alloca-tions, (c) data block allocations, (d) processor allocations, (e) memoryallocations.

Figure 10 reflects a modular design flow based on user requestsfor any of the reports for a given partition j of candidate configurationi. This particular report generation capability should be implementedfor access from batch job control, special user codes, as well as inter-active displays to obtain maximum evaluation flexibility to automati-cally recall and/or print alternative partition solutions for a givencandidate.

Specific output report formats are presented in Appendix B. Thedesign demonstration, Appendix C, has sample output reports for userreference.

3.3 FEASIBILITY DEMONSTRATION

In deriving a meaningful, yet simple, sample problem, specificpreliminary design material was obtained from Williams AFB with regardto an ongoing expanded design for the Advanced Simulation for PilotTraining multiple processor visual computational support subsystem. The

48

W F1

Aa a_ &t W

Ing D 1- 0 'j

Blate-C

* ! I zi

1- 0

0 c . 1 Li -~ox -

-Ihi l

0(1

1 4)I U go

L

49JS_.

0A

preliminary design material provided a realistic source of the formatfor ongoing trainer computational design input. It also included a mix ofgeneral-purpose and special-purpose processors. The information in thismemorandum provided a good base for generating a sample problem; how-ever, the resulting sample problem required simplification of the con-figuration described to permit a flexible, yet easy-to-follow, manualdemonstration problem to be obtained.

The design factors in the original problem were very restrictiveas to Central Processing Unit (CPU) task assignments and thus left verylittle room for alternative partitioning. This reinforces the factthat, in software design, tasks tend to be defined in terms of theselected hardware configuration features to meet computational needs, asopposed to specifying application computations and then matching tasksto the hardware selection. For the partitioning algorithm to be applica-ble to alternative allocations and partitions, the major feasibilityissue concerns design language and means for inputting the problemdefinition from which the partitioning model is to operate. These issuesare discussed in Section 4.

For demonstration purposes, overview inputs, restrictive inputs,and detailed inputs have been incorporated to illustrate various aspectsand paths of the partitioning process and to point out the tradeoffs inutilization of detailed inputs versus general estimates. The completealgorithm feasibility demonstration is included as Appendix C to thisreport. The basic order is the sample problem def-Inition, user inputsheets, user input echo summary, basic partitioning priority calcula-tions, sample performance feedback contingcncies, and solution summaryoutputs.

Figures 11 through 13 illustrate the major partitioning compon-ents as extracted and simplified from a set of Williams AFB ASPT prelimi-nary design notes for the visual subsystem. The overall processor con-figuration is denoted in Figure 11. The memory and external communica-tions are illustrated in Figure 12 to include both private and sharedmemory devices. It also includes processor-to-processor direct datatransfer. Figure 13 denotes the simplified task flow used for demon-strating the input and output steps of the algorithm. The tasks ofFigure 13 may be further divided into more detailed tasks for demonstrat-ing and testing specific features of the partitioning algorithm, once anautomated version of the algorithm is implemented.

The sample demonstration (delineated in Appendix C) permits thedefinition of potential automated implementation processes for handlingreal-world partitioning problems. The examples demonstrate the feasi-bility of an automated tool. Section 4 provides recommended implementa-tion steps for verifying and validating the partitioning tool. Thesesteps will require that the basic algorithm be automated to properlyevaluate and demonstrate its performance characteristics for more rea-listic partitioning problems that tend to be of larger size than the

50

a. xw

fI

laI~

enr~~IaII j~ > .

a - ----- - - a.

E

LL.

A 'A

14

W C

II 0UA CC

~II cc96 90wq

[BOOII us L

z 1..

ccoMA5

Z>=Ot

0- fcr La Z .AI x C>

-. j ILO

Q uIIk-

us IR

f0

UJ uj ZC0

LLa

2z

.jI<0 CLI.

manual demonstration examples. The manual examples will permit thebasic logic to be verified for a controlled, small-scale applicationprior to "cranking out" large-scale partitioning problems. This willpermit an initial level of confidence to be established in the automatedversion.

54

t .- --.I-..-,

4. MODEL IMPLEMENTATION CONSIDERATIONS

To successfully implement the software partitioning algorithm,an up-to-date technology data base for the flight training simulatorcomputational devices is essential. This section delineates the datacollection process and decision steps recommended for potential automa-tion and quality control of the algorithm defined in Section 3. Thissection has been organized to go from an overview of the candidate designevaluation environment into a detailed evaluation support data baserepository description, followed by computer selection criteria and therecommended implementation schedule for automation of the softwarepartitioning evaluation algorithm.

4.1 FLIGHT TRAINING SIMULATOR EVALUATION ENVIINMENT

Typically, the development of flight training simulator candi-date designs for the Air Force are contracted out by the SimulationSystem Program Office (ASD-SD24). The computational subsystem designdevelopment is monitored and evaluated by the Deputy of EngineeringSimulation (ASD-EN). In some cases, the flight trainer development isdirectly contracted by a specific system office (such as in the case ofthe F-16 trainer). Currently, the contracted organization has the pri-mary responsibility for establishing both hardware and software require-ments of the computational system, subject to certain Air Force guide-lines and training capability objectives. The candidate design evolvesthrough an iterative refinement of documentation and algorithm enumera-tion analysis, which typically progresses from system specificationfunctional flows followed by the detailed enumeration of the candidatedesign. Each of these levels has narrative descriptions interspersedwith a variety of technical charts, drawings, tables, flow diagrams,interface definitions, etc.; however, as denoted in Figure 14, thevolume of documentation for a training simulator quickly becomesunwieldy unless documentation traceability and content standards areadhered to and enforced via constructive reviews, which are geared todetecting and correcting errors early in the development phase.

This effort has specifically addressed the software partitioningaspects of candidate design evaluation. The three major outputs of thepartitioning algorithm are measures of the processing load balance,memory utilization, and estimated development implementation cost basedon given timing and sizing input requirements of the respective tasks anddata load for a given candidate configuration. For effective use of thesoftware partitioning algorithm, the underlying mathematical model ofSection 3.1 must be understood in terms of the processor utilization,memory utilization, and development cost formulations, which are theprimary outputs.

55

. .i

* COORDINATION -CONTROLS

SUBSYSTEM INTERFACE SPECSe COMMUNICATION PRIORITIES@ DATA FREQUENCY AND FORMATSs FUNCTIONAL DESCRIPTIONS

SUBSYSTEM DESIGN DOCUMENTSCREW POSITION - AIRCRAFT INSTRUMENTATION

CONTROLS - SWITCHES -ELECTRONICS

HYDRAULICS - WEAPON SYSTEMS - DISPLAYSAUDIO - VISUAL - MOTION - FORCE - NAVIGATION

TERRAIN - INSTRUCTIONAL OPERATIONS - SCORINGCOMPUTATIONAL ...

Figure 14. Hierarchy of flight trainer documents, which relates to candi-date design evaluation, can quickly become unwieldy if contentand traceability standards are not adhered to or enforced.The simulator computational subsystem interfaces with andcoordinates a large number of the trainer simulator sub-systems.

56

To obtain reliable outputs, a consistent, systematic procedureneeds to be established with appropriate configuration management andquality assurance provisions and controls. The major implementationconsideration for such a procedure is the establishment of a consistentdata repository for pertinent flight trainer computational design data.No central repository for Air Force flight trainer computational designscurrently exists, although various organizations (such as ASD-EN) dohave their own evaluation data repositories.

During the course of this contract, it was learned that the NavalTraining Equipment Center (NTEC) in Orlando, Florida, does have arepository of all documentation associated with Navy training devices toinclude the computational subsystem. NTEC recently modified therequired Data Item Descriptions related to the computational subsystemto be an integral part of training device development in conjunction witha proposed Appendix A to the trainer specification, MIL-STD-1644,entitled "Trainer Software Design, Control, Production Testing andAcceptance Procedures and Requirements." This proposed specificationincorporates the top-down structured design approach with minimm stand-ards that are required of each milestone document and its associatedreview content, error detection/correction actions, and milestone com-pleteness determination. The procedures are in basic agreement with thedevelopment cycle presented in Section 2.1. This set of documents per-mits a consistent repository to be established and maintained for cur-rent reference and analysis input for new development considerations.Unfortunately, it is still primarily a manual information storage andretrieval system when it comes to accessing data pertinent to softwarepartitioning.

The factors identified in Section 3.1 that influence optimalsoftware allocation (such as: data block, task, processor, and memorydescriptions) remain the same regardless of the system assumptions orpresentation format. Indeed, these factors (Table 4) must typically beextracted from more than one document to obtain the complete set of inputand constraint parameters defined in the mathematical statement ofSection 3.1. To assist in the review of documents with respect tosoftware partitioning of the computational subsystem, the supportingdata base parameters have been segmented into five major areas withrespect to flight trainer simulator:

1. Trainer Computational Interface Requirements2. Baseline Application Components3. Candidate Hardware Configuration Components4. Technology Data Base5. Evaluation Criteria/Constraints and Partitioning Load.

Figure 15 reflects the interactive nature of these data base areas withrespect to technology capabilities and the development cycle up throughthe completion of the design but prior to actual implementation and test-ing. The upper area relates to milestone documents of the training

57

..

TABLE 4. DEVELOPMENT DOCUMENTS AND THEIRRELATIONSHIP TO THE PARTITIONINGALGORITHM FOR SOFTWARE SYSTEMS

DOCUMENT(S) INPUT AREA

Computational Subsystem External Device InterfacesInterface Specification

Required Components

Functional I/0 Map

Comnunication Rulesand Priorities

Baseline Load(s)

Software Design and Data Block DescriptionsData Base Specifications

Task Descriptions

Task Threads

Baseline Load(s) Tasking

Hardware Configuration ProcessorsDesign Specifications

Memories

Interfaces (Internal andExternal)

Communication Rules

58

....

SPECIFIC TRAINER DEVELOPMENT

COMPUTATIONAL I CANDIDATEINTERFACE & I SOFTWARE &.FUNCTIONAL I HARDWAREREQUIREMENTS SPECIFICATIONS

HUE AR T

IS COMPUTATIONAL 0U CANDIDATEN

P ~~DESIGN - - - - -

PARTITIONING &LEVALUATION

MS TRAINING, MEMORIES

INSTRUCTOR, IPROCESSOAS.& OPERATION I& COMMUNICATIONDEVICE IDEVICE SINTERFACES IINTERFACESS

TECHNOLOGY CAPABILITY

Figure 15. Computational design evaluation must relate a specific designin terms of current technology capabilities for both externalcomhiunicatlons and internal computational subsystem details.

59

computational interface requirements, software design, and hardwaredesign respectively. The lower half represents the technology database, which permits an abbreviated means for entering the design detailson which the partitioning algorithm is to operate. The left half relatesthe devices to be serviced by the computational subsystem, and the righthalf reflects the internal computational subsystem structure organiza-tion and devices.

Although the data are extracted from independent sources, it re-quires interactive coordination and configuration controls to ensurethat accurate, up-to-date, best estimates are utilized for the evalua-tion at hand. The evaluation criteria and constraint inputs facilitateconfiguration controls, parametric analysis, and partitioning flexi-bility with respect to prohibited and/or preassigned allocations inaddition to initial allocations. The details of this segmented data baseare now described in terms of implementation considerations.

4.2 DATA BASE INAGEMENT

Two major recommendations are being made to facilitate orderlyconsolidation of the storage and retrieval for each of the five data baseareas that provide the driving source of information for the partition-ing algorithm and candidate design evaluation process. These recom-mendations are as follows:

1. The addition of a standard set of candidate design specifi-cation tables that address the software and hardware designsas independent sets of parametric measures.

2. The establishment of a design evaluation data base reposi-tory utilizing an interactive file management system underthe configuration control of ASD/ENETC.

This subsection supplies key factors that should be evaluated and modi-fied as necessary to facilitate an orderly transition to an automatedalgorithm implementation as presented in Section 4.4. Proper utiliza-tion will require a training indoctrination as to the potential benefitsto both the flight trainer developer and evaluator communities. Beforethe recomended input forms are described, several master data struc-tures are delineated that have a direct influence on validity of dataentries and provide the key to independent software and hardware designcharacterization.

4.2.1 Master Data Structures

These mester structures include (a) data block characterization,(b) memory characterization, (c) task characterization, and (d) proces-sor characterization.

60 _-

* '.~ &. .;-4~N

Combinations of these structures are incorporated into therecommended forms for each of the five data base input areas presented inAppendix A.

4.2.1.1 Data Block Characterization - Data characteristics suchas source, volume, frequency, content, and destination are the real-timedrivers of the computational subsystem from both external device and in-ternal task communications, command, and control. Table 5 denotes at-tributes required by the software partitioning algorithm for each datablock that is acted upon or created by the computational subsystems beingpartitioned. Note that these attributes do not tie the data block to aspecific storage device. Only external system blocks are identified asbeing related to a given type of peripheral interface; for example, acockpit control setting input buffer block has a definite source devicethat must be monitored at a predetermined sample rate. On the otherhand, the data to be computed by one task and used by a sequentiallydependent task are described in terms of minimum storage device require-ments for their storage and retrieval utilization. These master blockdefinitions are then referenced by the block identification when refer-enced in the task descriptions (Section 4.2.1.2) or in evaluation allo-cation restrictions (Section 4.2.3).

4.2.1.2 Memory Characterization - A wide variety of memoriesmay be incorporated into a candidate design configuration for a flighttrainer. For purposes of partitioning, memories are categorized (as de-noted in Table 6) to include read-only memory (ON), writable controlstores (WCS), main random access memory (RAM), rotating random accessme ory (RRAh), and sequential memories (SM). Within each of thesecategories are additional retrieval and storage characteristics for datarepresentations of addressable units. These representations permit thegeneric data block parameters of Section 4.2.1.1 to be matched withappropriate memory devices in the candidate configuration for whichpartitioning is being performed.

4.2.1.3 Task Characterization - Specification of task attri-butes, which are independent of the processing hardware, poses a verychallenging problem area for incorporating the traditional hardware-dependent design customs and notations that have evolved not only inflight training simulator design but computational system designs ingeneral. At this point in software design history, several emergingphilosophies for design standards seem to be contradictory concerningthe level of specification and the documentation language used to conveythe detailed software algorithms to be implemented. At one extreme isthe use of English-like structured pseudo code, which is favored for itsfeatures of being easy to follow and comprehend. On the other hand,there is an emphasis for precise, unambiguous mathematically enumeratedrepresentations that provide the specific computations but, if notannotated with English descriptions, they become very hard to follow,except for persons who are very familiar with the specifics of thealgorithm. Most designs are generally a mixture of these two approaches,

61

TABLE 5. DATA BLOCK CHARACTERIZATION

ATTRIBUTE VALUES UNIT/MEANING

Identifier 6-Character Provides a unique identifierMnemonic for cross-reference and

labeling purposes

Level I Character

" I's System Interface• )G Global (used by more than

one task)= 'LI Local to one task but must

be saved" 'T Temporary scratch area for

a given task

Discipline 4-Character Code Provides basic 1/0 requirementfor determining suitablememory device allocation

" 'FIFO' Queue" 'LIFO' Stack" ISEQ' Sequential" IRAN# Random" 'ROR' Ready-Only Random" "ROS' Ready-Only Sequential" 'CBUF' Circular Buffer

Sizing

* Maximum Records Positive Integer Records* Bits/Charac- Positive Integer Bits

ter* Characters/Word Positive Integer Bytese Average Words/ Positive Integer Words

Record* Maximum Words/ Positive Integer Words

Record* Minim.m Words/ Positive Integer Words

Record

62

TABLE 6. MEMORY DEVICE CHARACTERIZATION

ATTRIBUTE VALUES UNIT/MEANING

Identifier 10-Character Provides a unique identificationMnemonic for each memory device in the

technology data base for whichthe following attributes define

Type 4 Characters

a 'ROM' Read Only Memory. 'RAMM' Random Access Main Memorya 'RRAM' Rotating Random Access Memory. ISM' Sequential Memory.'WCS, Writable Control Store

Size in Bits

* Minimum Positive Integer Bits* Maximum Positive Integer Bits* Increments Positive Integer Bits

Number of Positive IntegerDifferentAddressable Units

For Each

Addressable Unit

* Level 4-Character Code

" 'BIT' Bit Addressable" '6BB' 6-Bit Byte Addressable" 'BB' 8-Bit Byte Addressable" 'WORD' Word Addressable

s Bits/Unit Positive Integer Exclusive of Parity or ErrorLevel Deletion Correction Bits

s Read Access Real NanosecondsTime

* Read Cycle Real NanosecondsTime Unit

* Maximum Positive Integer Same as Unit LevelSequentialUnits Trans-ferred forSingle Read

* Write Access Real NanosecondsTime

63 tI

TABLE 6. MEMORY DEVICE CHARACTERIZATION (Sheet 2 of 2)

ATTRIBUTE VALUES UNIT/MEANING

a Write Cycle Real NanosecondsTime/Unit

* Maximum Positive Integer Same as Unit LevelSequentialUnits forSingle WriteAccess

* Error Detection/ 6-Character CodeCorrection z 'PARITY' Parity Bit

s 'SECOED' Single Bit Error CorrectionDouble Bit Error Detection

Number of Sup- Positive Integerpliers for EachSupplier

s Identifier 10 Characters Unique Identifier

* MTBF Real Hours - Mean Time BetweenFailures

* MT1R Real Hours - Mean Time to Repair

* MSPM Real Hours - Rescheduled PreventiveMa i ntenance

# MTPM Real Hours - Mean Time fur Preven-tive Maintenance

64

64

, . .. .. .

which facilitates the overall functional flow, high-level presentationand permits a traceability structure for enumeration of detailed designcomputations and decision logic.

The remaining problem area of design specification relates tothe specific notation. Certain aspects of flight trainer computationalalgorithms have become well-defined, i.e., aircraft flight kinematics.These algorithms are generally used for making benchmarks on new candi-date processors. Thus, for well-established algorithms, a master set ofsimulation task benchmarks can be established for each candidate proces-sor being considered. New algorithms require a more fundamental break-out of the instruction mix to ascertain timing and sizing elements. Insummary, a master set of software task attributes are presented inTable 7. The establishment of a master instruction mix, task I/Odescriptors, and task enablement features is recommended as one of thesteps (Section 4.4) toward algorithm implementation. Related to thismaster instruction mix is the development language for task code genera-tion. Recent trends in simulator coding have incorporated FORTRAN codefor the scientific mathematical application models, but there is still astrong dependence on the assembly level code for expressing real-timeexecutive and I/O handler modules to meet the real-time timing require-ments. The selection of a task design instruction mix notation should becoordinated with the simulation high-order language efforts and proces-sor instruction architectures.

One way to obtain this information would be the use of a graphi-cal task flow representation, which included a standard design notationto indicate the instruction sequences, loops, and relationships withI/O. A flow notation, such as TBE's Input/Output Relationships andTiming Diagrams, can be automatically traversed with the instruction mixand I/O features being identified and reformatted for use with the parti-tioning algorithm. This would require that a standard flight trainercomputational design language and flow representations be established,thus providing a standardized way for documenting the detailed taskcomputational designs.

An important note is made here regarding the traditional meansof expressing task sizing and timing in terms of adds, multiplies,branches, etc. The instruction mix need not be at the machine level.Instead, it should reflect a set of simulation macros, such as singlevariable linear table interpolation, and trigonometric functions. Eachof these, in turn, is characterized for each candidate processor as totiming and sizing. If the simulation macro has been implemented infirmware or as part of a mathematical package, the sizing is reduced interms of the main instruction storage for the task.

4.2.1.4 Processor Characterization - Processor technology isconstantly expanding in terms of operating system and instruction setcapabilities. Table 8 lists processor attributes that pertain directlyto the software partitioning algorithm. The operating system features

65

TABLE 7. TASK CHARACTERIZATION

ATTRIBUTE VALUES UNIT/MEAINING

Identifier 6-Character Provides a unique identifierMnemonic for cross-reference and

labeling purposes

Source Language 10-Character Must match entry in theCode master source language

list maintained for currentprocessor technology

Instruction Mixfor Each InstructionType:

* Instruction Iden- 10-Character Must match entry in mastertifier Code simulator instruction mix

identifiers

* Sizing Count Positive Integer Number of times this Instruc-tion appears in code

* Execution Count Number of instruction inter-Average Positive Integer actions considering looping

conditions for average andWorst Case Positive Integer worst-case logic

ata Retrieval forEach Task Input

# Block Identifier 6 Characters See Table 5

* When 6-Character Code

x 'START' All records read at firstof task before main proces-sing

- 'ALONG' Records processed one ata time

* Average Input Positive Integer Records

# Minimum Input Non-Negative RecordsInteger

@ Maximum Input Positive Integer Records

66

TABLE 7. TASK CHARACTERIZATION (Sheet 2 of 2)

ATTRIBUTE VALUES UNIT/MEANING

Data Storage for Each

Task Output:

e Block Level 1 Character See Table 5

e Block Identifier 6 Characters See Table 5

e When 6-Character Code

a 'ALONG' Records are output via indi-vidual processing

a 'END' Records are output just priorto task exit

e Average Output Positive Integer Records

* Nlnimum Output Non-Negative RecordsInteger

* maximum Output Positive Integer Records

Enablement

* Type 4-Character Codea -TIME' Time Enabled, DATA, Data Enableda 0SLVD, Slaved to Master Task- ITAD' Time and/or Data Enabled

o Frequency 1 Real Iterations/Second for TimeEnablement

* Frequency 2 Real Iterations/Second for DataEnablement

a Frequency 3 Real Iteratins/Second for Slaved

67

-" - ¢ ...... ." .. - "- ' P ;

. .1 . -- .........

TABLE 8. PROCESSOR CHARACTERIZATION

ATTRIBUTE VALUES UNIT/MEANING

Identifier 10 Characters Unique identifier for pro-

cessor with the followingattributes

Operating System

0 Multitasking

A Levels Integer.GE.1

A Numer of Integer These many levels are ser-Priority Levels .GE.O viced in a priority fash-

.LE. Levels ion. The remaining levelsare serviced in a circulartime-shared fashion.

e Enablements Integer Enablements/Second

A Maximum TimeEnablementFrequency

A Resource Fig.9.GE.0 MicrosecondsManagement perTime Enablement

A Max'! xn Data Integer Enablements/SecondEnablementFrequency

A Resource F10.9.GE.0 MicrosecondManagenmt perData Enablement

A Maximum Slaved Integer Enablments/SecondEnablementFrequency

" Resource F19.9.GE.0 MicrosecondsManagement perSlaved Enable-"wit

68

No-WNWa

• ...• .... ' '-: - - -' - . ;,. .: , ;' ' - , .... . .

TABLE 8. PROCESSOR CHARACTERIZATION (Sheet 2 of 3)

ATTRIBUTE VALUES UNIT/MEANING

9 For Each TaskLevel LA Maximum Number Integer .GE.1of Task Level L

ATask Service CodeScheme forLevel L

a p Prioritya 1C. Circular= OF' First-in, First Out

e Level Resource F10.9 .GE.9 MicrosecfhdsManagement

Simulation InstructionSet Measurements forEach BenchmarkInstruction 1

e SizingMeasurementsA Number of Code

MemoriesInvolved

The Memory Type for 4-Character Code Must agree with masterEach Code Memory m memory types defined in(the first memory is Group 4the user task code --any other memories arepredefined for thisprocessor)

£ Length of Code Integer .GE.1 Number of basic units usedin Memory m to describe memory m (see

Group 4)

69j

TABLE 8. PROCESSOR CHARACTERIZATION (Sheet 3 of 3)

ATTRIBUTE VALUES UNIT/MEANING

a Timing Measurements kul Implies Averagefor Each Code k-2 Implies worst CaseMemory m and lcal,2

A Number of Scratch Integer .GE.0Data Store Waits

A Number of Scratch Integer .GE.0Data Store Waits

AComputational Integer .GE.0 CyclesTotal for AllMemories

a Application Develop-ment MeasurementsUsing Language L ofthe Master LanguageListA One Item Develop- Integer Man-hours

ment ChargeA Change per Appli- Integer Man-hours

cation Instructionof this Type

70

s-. , - 0 A

applicable to software partitioning relate to multitasking disciplines,limits, and resource management services. The instruction set ischaracterized in terms of the master simulation instruction set asdescribed in Section 4.2.1.3, along with attributes for user memory i/0versus preprogrammed resources plus development cost estimates.

4.2.2 Suggested Input Forms

The forms, as designed, may be used directly by a data keyingoperator to produce keypunched cards or entry directly onto a file via aninteractive data entry terminal. Specific physical file formats are notspecified since they will be a function of selected computer file imagecapabilities described in Section 4.3. Because of the volume of inputsheets, they are presented in Appendix A for each of the data base files.

During the design of the input forms, emphasis was placed onconsolidation and cross-reference techniques that facilitate an organ-ized straightforward user input interface. The software partitioningalgorithm requires an assortment of specific data to fully definetrainer system interfaces plus computational hardware and softwaredesign details that must be accurate if a good partition allocation is tobe obtained. The separation of forms is based on the five major inputareas, and it is recommended that these areas be standardized for pre-senting the respective interface requirements, software task/data designrelationships, candidate hardware design configuration, technology capa-bilities, and evaluation priorities, including the candidate initialdesign allocation as a starting point for partitioning optimization.

4.3 TARGET COMPUTER AND SOURCE LANGUAGE SELECTION

The selection of the computer system for the partitioning algo-rithm should consider, as a minimum, the following features, which mustbe incorporated to facilitate automatic implementation of the partition-ing algorithm and its potential expansions:

1. Data base management system2. Structured program language3. Modified linear mixed integer program optimizer4. Computational speed and accuracy.

Each of these features is described in more detail in the follow-

ing paragraphs.

4.3.1 Data Base Management System

The interrelated, yet separate, data files (described earlier inthis section) of the recommended flight trainer automated repository arebest implemented under a standard data base management system that

71

A,.-.A.

permits creation, update maintenance, and configuration management ofall data and program files. It is recommended that system data filemanagement utilities be available to the user in several differentmodes, including batch job control, interactive terminal commands, anduser program code directives to permit a flexible, yet controlled, dataaccess environment. Direct record access capability is an essentialfeature for implementation of the software task and block descriptionplus the technology data base files.

The amount of data is a function of the flight training simulatorcomputational candidate designs to be evaluated. Table 9 provides anabbreviated summary of sizing relationships for each record type groupcontained in the respective files required for the partitioning algo-rithm. The data base management should include memory management of codeand data required for execution. Internal tables utilized by the algo-rithm are sized in Table 10. The algorithm code is estimated to be10,000 lines of structured FORTRAN exclusive of potential data managerand optimizer extensions.

4.3.2 Structured Program Language

Evaluation code (code used to facilitate manual analysis) is avery useful tool if it can be maintained under configuration control andpermit expansion to more detailed models when necessary for a givenevaluation analysis. Structured source code facilitates modularity and,thus, permits model expansion. Several source languages are includedhere as candidates for the partitioning algorithm implementation,including FORTRAN 77, JOVIAL, and ADA. These languages were selectedbased on current DOD-approved languages and language development activi-ties. Pros and cons for each are now presented.

The widespread recognition of FORTRAN for scientific and mathe-matical programing makes it the preferred language of the three lan-guages considered. The newest ANSI FORTRAN 77 standards incorporatecharacter manipulation, which is independent of machine architecture.Its use of structured logic includes both true and false process defini-tions without the use of extraneous "GO TO's." File manipulation capa-bilities have also been expanded to include file status checks andstandardization of certain types of data storage/retrieval mechanismsthat have previously required vendor-peculiar FORTRAN extensions. Someproblems may be encountered with new compilers being released to meet thenew FORTRAN standards, but these compilers should evolve rather quicklyto support most of the ANSI 77 features. This will result in code thatis more easily transported from one machine to another. This is animportant aspect, since the partitioning algorithm does not require adedicated computer system, and as such, it is envisioned as being auseful tool for flight training simulator developers and maintenancereconfiguration analysts, as well as for Air Force evaluators. Each ofthese specialists generally has his own in-house computer systemtailored for specific analysis needs.

72

-AUT

iM

L..MI Lla~ '

.4 Z

LU

W6

La.

16-0

- &A AL 73

4, 4, 'U 6

CYC

us 4-4 4

a I. I

to U. 0- b w

E. I ~ I I (je

L L

LL 00 ciL

Cie0

'A'

U. low

74-

4J

1~-L . C.NCl

4J 1.

cc c .-

e) L

*-i0-4-

LIco 2~

LU - ~ ~ 75

TABLE 10. INTERNAL ALGORITHM TABLE SIZING REQUIREMENTS

TABLE WORDSIPT NO. TABLE TITLE (60-bit words)

1 Limits, Constants, and Code 20

2 Current Problem Sizing Controls 9

3 Priority Controls 28

4 Current Processor List P*(13+i)

5 Current Memory List 11*M

6 Current Communication Link List (3+3*QND)*9

7 Current External Device List (4+DB)*d

8 Task/Processor Allocation and 9*T*PRestrictions

9 Memory/Processor Communications (4+4e)*M*PAllocation and Restrictions

10 Memory/Block Allocation and 5*M*BRestrictions

11 Master Block List (11+M+2T)*B

12 Master Task List (16+5i+6*B+e)*T

13 Scratch and Local Parameters To be Defined

76

, &Ld"

JOVIAL is mentioned because of its recognition by the Air Forceas a standard language for embedded computer systems development. Amajor drawback is its limited I/O capabilities, which is a major factorwith regard to the partitioning algorithm's large data base handlingrequirements.

ADA is also mentioned since it is the DOD language being

developed with source language standardization as a major goal to sup-port software development of new military computational subsystems. The

on-going compiler developments are limited to experimental compilers andcompiler design efforts. Therefore, at this time it is not a feasiblecandidate for actual algorithm development and testing. It will be 2 to

3 years before it is available in an operational development setting.Further implementation/expansion should monitor and consider ADA sinceits features will permit more configuration control as well as the struc-tured expression of concurrent process control flows, I/O, and computa-tions with concise data base definition.

In conclusion, FORTRAN is the recommended language for imple-

mentation of the partitioning algorithm.

4.3.3 Modified Linear Mixed Integer Program Optimizer

The partitioning algorithm has the potential for future inter-faces with a modified linear program mixed integer program optimizer.The current algorithm design is based on a heuristic algorithm driverthat assumes that an initial feasible partition exists with respect tothe basic real-time processing requirements of data availability, tasktiming, and less than 100% processor/memory allocation. From thisinitial feasible solution, it seeks to determine and make improvementson the initial partition with respect to three goals: (a) processor loadbalance within given growth allotments, (b) memory utilization withingrowth tolerances, and (c) minimization of development costs. Althoughheuristics do not guarantee an optimal solution, it is anticipated thatthe complexity of priorities and data constants will change frequently,which makes the finding of the true optimal a meaningless exercise.However, optimizers can be employed to help find an initial feasiblesolution and to find optimal subset solutions under the control of theheuristic decision tree. In the case of the partitioning algorithm, theinitial feasible solution poses the largest problem in terms of sizingand numeric accuracy techniques that are required. Table 1 summarizes

t the optimizer sizing as a function of the size of candidate designs to beevaluated.

4.3.4 Computational Speed and Accuracy

Although the partitioning algorithm is not as demanding as real-time simulation or control codes, it is important that it be able tosupport quick-turnaround evaluation runs to expedite the given evalua-tion case. The complexities of the processor utilization calculations

77

.. .

in terms of task computations, resource management, and I/O are iteratedwith respect to potential processor tradeoffs for load balance calcula-tions that involve a variety of attributes. Since the basic computationsare subject to mathematical model expansions and changes, floating pointcapabilities are recommended to permit new equations to be introduced,as required, without the burden of fixed-point scaling.

Units have been selected to keep related variable numeric orderof magnitudes within computational limits of most scientific machines.These units should be periodically examined as technology advancementsare made. For example, many current real-time flight trainer applica-tion cycles are based on 1-sec intervals with subcycles or subframesmeasured in terms of milliseconds. As timing improvements are made,these may take on smaller increments of time for application cycling,hence the need for their periodic reappraisal. Another factor is machinecycle time, which is currently measured in nanoseconds; thus, certaincalculations involving memory I/O must be accumulated separately toobtain totals that can then be used to determine any appreciable I/Otiming for tasks that handle large volumes of data in addition to compu-tational processing. Typically, 32-bit floating point can represent sixsignificant digits. Thus, if a basic unit is assumed to be 1 sec, thenanosecond effectively is disregirded unless accumulated separately.However, if either double precision (64 bit) or 60-bit single precisionis used, there is no problem. An alternative is for task memory I/O,resource management, and individual instruction timing computations tobe accumulated for total task time in microseconds, and then task timesmay be added separately for a given application cycle time in terms ofcurrent task/cycle relationships. Thus, there is the need for floatingpoint, with a minimum of 32-bit words sufficing for most operations, andeither segmented units or double precision variables to account forapplication subtask timing computations.

The use of preemptive priorities rather than weighted prioritiespermits processor loading, memory allocation, and development costs toremain in their standard units without any input scaling and outputrescaling. However, in each priority level, numbers for a given task ordata block should be summed separately from totals being used for totalmemory or total processor utilization to avoid underflow accumulationproblems.

4.4 RECOMMENDED IMPLEMENTATION SCHEDULE

The major tasks and their hierarchical relationships aredepicted in Figure 16. Each of these tasks is briefly described in thissection with cross-references to appropriate report sections for relateddetails. Although some parallel task sequences are depicted, there aresome interdependencies, as denoted in Figure 16. These interdepend-encies are basically handled at major detailed reviews, which are recom-mended to be held quarterly to assess the implementation progress, to

78

.t

ESTABLISH DESIGN CODENERIFY DESIGNMODEL REPOSITORY BASICOPIZEVALIDATION PROGRAMS ALGORITHM PROGRAMS

PROCEDURE PROGRAMS PROGRAMS

VALIDATE VORMALFA

EXPANDED ACCEPTANCE RPRMODEL TESTING

Figure 16. Algorithm implementation tasks.79

ensure that interface definitions are adhered to, and to establish moredetailed interfaces as the appropriate operational consideration detailsbecome known.

Figure 17 groups the tasks into four major implementation phasesover a 2.5-year period. There is an overlap between Phase III and PhaseIV, with the major emphasis of Phase III placed on basic (as currentlydesigned) algorithm validation and with Phase IV emphasis on an expandedvalidated model incorporating an optimizer for selected aspects of thepartitioning algorithm. The implementation tasks are now described byphase. To make a complete task statement, there is some redundancy withearlier report sections. Cross-references are made to avoid excessiveredundancy.

4.4.1 Model Validation Plan and Selected Computer Interfaces

Although the candidate computer selection aspects have been de-scribed (Section 4.3), the specific computer implementation must be fur-ther delineated to obtain a practical partitioning allocation and eval-uation tool for flight trainer simulator computational candidate design.Existing evaluation computer facilities should be reviewed for currentformats and data collection procedures in addition to the current com-puter capabilities to contribute basic inputs to the Phase I tasks, whichare now briefly described.

4.4.1.1 Validation Plan - The sample problems manually demon-strated under this contract have verified the feasibility of the parti-tioning algorithm design. However, they do not constitute a model cali-bration case from which a confidence level of model validity may bederived. As evidenced in the mathematical statement of the partitioningproblem (Section 3.1), there are many interrelated variables and factorsthat drive the partitioning process, necessitating some parametric auto-mation techniques to fully analyze the automated design validity andstability for real-world data. The validation plan will permit con-trolled algorithm implementation testing to determine its validity withrespect to known partitioning situations of selected flight trainingsimulator computatational designs. By addressing evaluation partition-ing problems to be handled prior to algorithm coding, the evaluationcommunity is essentially establishing the foundation for the algorithmacceptance test with respect to its role as an evaluation tool.

As a minimum, the validation plan should identify the flighttrainer system(s) to be used as the algorithm implementation baseline.It should also extrapolate intended sizing of the algorithm applicationin terms of the number of each data base item described in Section 4.2(i.e., number of tasks, blocks, processors, memories, etc.). A set oftest cases should be drafted in an outline format as to specific algo-rithm features to be incorporated and tested for both the basic model andthe expanded model.

80

ell 1 1

0 0 1em I I2 l II6

0.0

Z Cl. I 0 iJa)-a

2j L4 0

ZA Zu z 0a -, -=z iL

W x J Cw 2 z UZJIZ ia w(O

00

a.. - > u 0 ILI W0 9-0

SP . 0 C

LU~q -J I..

>0r 10 a. -U- 1

ta 1~ - *-<~~~a LC > 3<I

a a § cc '

-dellellll' I I-,

:oz' VCI- C enI.-I 1

816

4.4.1.2 Data Base Interface - The specific flight trainer com-putational design repository format and data base management utilitiesshould be delineated by this task. This includes finalization of theuser interface formats (such as those contained in Appendix A) and theformat by which the partitioning algorithm may retrieve its inputs andstore its outputs with respect to the repository and the interactiveand/or batch user.

This task incorporates the data collection, storage, andretrieval mechanisms, plus quality assurance steps necessary for algo-rithm implementation and usage. The repository data management shouldincorporate responsible agencies for each input area and make maximumuse of pre-editing and file management utilities of the selected com-puter system. The results of this task should be compiled in the form ofa users' manual for the flight trainer design repository and specifi-cally address the partitioning algorithm interfaces. These interfacesinclude the master design simulation language instruction set and guide-lines for processor, memory, task, and data baseline descriptions(covered in Section 4.2) that will streamline the orderly preparation ofinputs and permit gradual controlled growth into a fully tested andimplemented repository system for multiple evaluations.

4.4.1.3 Computer Selection - Computer candidate selection hasbeen discussed in Section 4.3. This task ties Phase I activitiestogether to determine the specific coding standards and interfaces to beemployed for algorithm implementation for a given computer facility.

4.4.1.4 Optimizer Interface - This task permits the long-rangeinterface goals to be defined for potential optimization steps in theheuristically driven partitioning algorithm. This is a major area forfurther study and, as such, is recognized in Section 5.3.

4.4.2 Automated Algorithm Verification and User DesignFoundation

Phase 11 permits the initial automation of the basic algorithmand delineates additional programs that will aid in the bookkeeping andincrease computational confidence levels of an expanded partitioningalgorithm. Each of the tasks is now.defined.

4.4.2.1 Establish Model Validation Procedures - This taskexpands and enumerates the test cases outlined in the test plan of PhaseI. The nature of the basic partitioning algorithm is to seek and, ifpossible,-find an improved partition of tasks. Thus, the test proceduresmust include the means for reconfiguring the subject flight trainer forwhich 4"upposedly "better" partition has been found. In addition,related performance measurements of the newly partitioned configurationmust be specified as to what and how they are to be collected andevaluated to access the predicted partition improvements of the parti-tioning algorithm. To assist in this step, the multiple processor

82

simulator being designed under separate contract may be used to provide aquick look at the dynamic aspects of the new partition prior to making areconfiguration decision. All of these considerations must be placedinto a timeline for algorithm validation testing to account for permis-sible reconfiguration in the partitioning restriction. For example, ifspecial-purpose tasks may only reside on special-purpose processors,they should be declared as such in the partitioning algorithm evaluationoptions. Thus, realistic, measurable validation test procedures are thegoal of this task.

4.4.2.2 Design Repository Programs - The users' manual of PhaseI will undoubtedly require specific repository storage/retrieval pro-grams to be designed to augment the system-supplied data base capabili-ties to support the flight trainer evaluators "input jargon" and toefficiently handle the input and subsequent updates to each of the vari-ous files to ensure consistency and completeness of any given repositorytransaction. The results of this task constitute the detailed design ofeach and all repository programs to be implemented in Phase III.

4.4.2.3 Code/Verify Basic Algorithm - This task is the moststraightforward of all of the tasks and simply entails the coding, debug-ging, and verifying of the basic algorithm as designed and demonstratedas part of this subject contract. This provides the working baseline forall future expansion in both model repository and optimizer interfaces.The results of this task provide a source code listing, verification testcase execution outputs, and documented interpretation.

4.4.2.4 Design Optimizer Programs - The emphasis of this task isto be placed on upgrading and complementing an existing mathematicaloptimizer package selected in Phase I with respect to computational andlogic needs peculiar to the partitioning application. This taskrequires extensive knowledge and experience with mathematical optimiza-tion codes and their numerical stability in terms of accuracy, scaling,iteration, and masking techniques that can judiciously expedite thesolution space search for initial feasible solutions. The task alsorequires knowledge and experienc;. with optimal subproblem solutions ascalled by the heuristic driver of the basic algorithm. The results ofthis task will comprise the detailed design of programsto be implementedto support the optimizer interface.

4.4.3 Basic Model Validation and Expanded Program InterfaceDevelopment

This critical phase permits the large-scale, real-world dataassessment of the basic algorithm to be made. The first part of PhaseIII is associated with specific data collection, scripting, and supportprogram coding. The latter part of this phase incorporates efforts ofthe first part for basic algorithm validation testing. In addition, theoptimizer programs are developed in preparation for the Phase IVexpanded model. Each of the Phase III tasks is now described.

83

4.4.3.1 Script Validation Data - Validation input data must becollected and prepared utilizing the validation input procedures foreach test case for basic algorithm and expanded algorithm validationcases. A test case can not proceed until its basic inputs have beenproperly prepared.

4.4.3.2 Develop Repository Programs - The programs designed intask 2.2 of Phase II are coded, debugged, and verified by means ofvalidation input procedures to assist in the input processing oftask 3.1.

4.4.3.3 Develop Optimizer Programs - This task codes and debugsthe programs designed in task 2.4 of Phase II in preparation for expandedalgorithm verification and validation of Phase IV.

4.4.3.4 Validate Basic Algorithm - Each validation test case ismade in the order prescribed in the test procedures. If any problems areencountered, their impact on the test plan and case procedures must befully evaluated to determine what action, if any, is necessary to con-tinue the test program. All test execution reports should be included asappendices to the test summary report. It is anticipated that certainvalidation tests can be run prior to complete implementation of therepository to exercise the fundamental paths of the algorithm.

4.4.4 Expanded Model Verification, Validation, and FormalAcceptance Testing

Phase IV paves the final path to the realization of the parti-tioning algorithm as part of the standard flight trainer simulator comp-utational design evaluation and/or design guide tool. The full reposi-tory and added optimizer capabilities developed in the first threephases are now integrated and tested to provide a controlled user iqter-face for multiple evaluation situations. The tasks are now defined.

4.4.4.1 Verify Expanded Model - This task consists of selectedbasic algorithm test cases to verify that these cases are still properlyhandled in the expanded model. In addition, new path verification testsare incorporated by the designer to verify that new capabilities areworking as designed.

4.4.4.2 Validate Expanded Model - This task performs the exten-sive testing as defined in the validation procedures for the extendedmodel. As with basic algorithm validation, if any problems areencountered, their impact on the test program must be evaluated and itmust be determined whether any action is necessary for continuance of thetest program. All execution results should be included as appendices tothe test summary documentation.

4.4.4.3 Formal Acceptance Test - The complexity of the parti-tioning algorithm and its potential evaluation decision-making impact

84

necessitates the need for formal Government acceptance tests. Thesetests should be scripted and performed by an independent organization to

fully assess the delivered capability with respect to completeness of

documentation, configuration, quality, and purpose. The major developer

is involved as a consultant to explain or expand documents and to respond

to any questions concerning the delivered operational package. It is

anticipated that Government flight trainer system evaluators will beresponsible for scripting and conducting these independent test proce-dures since the test will serve as a training task that emphasizes the

intended operational user environment of the algorithm.

4.4.4.4 Final Report - The emphasis of this task is to be placed

on finalizing documentation of the automated algorithm capabilities,findings, and conclusions. This documentation should be accompanied

with the final user, test, and program maintenance documentation forspecific program implementation details.

85

- .-. ,*

5. CONCLUDING REMARKS

Software partitioning is a complex, design development/evaluation, decision-making process with many tradeoffs to be analyzedfor selecting a good candidate flight training simulator computationaldesign for a particular operational trainer implementation or upgrade.This section briefly summarizes the details presented in Sections 2through 4 in terms of the study findings, related work, and areas forfurther study.

5.1 FINDINGS

Candidate software designs expressed independently of candidatehardware are the basic key design feature that permits software parti-tioning flexibility. This is not the traditional design approach cur-rently in use for system design. This project has defined the types ofdesign data that will permit independent assessment of baseline softwaretasks for alternative multiple-processor configurations. The key dataareas are the establishment of a standard design language and an auto-mated repository for the given application design data.

The partitioning algorithm has been designed as a general parti-tioning algorithm for software systems, and it is the data collectionprocess (Section 4.2) that will make this algorithm unique for a givenapplication implementation. In this way, it is seen as a useful tool forthe evaluation of a wide variety of computational subsystem designssince it is not constrained to current configuration, technology, orapplication.

5.2 RELATED WORK

The results of this effort are closely coordinated with ContractNo. 733615-79-C-0003 for the AFHRL Advanced Multiple Processor Configu-ration Study. The multiple-processor study is concerned with featuresand techniques for assessing the predicted performance of given alterna-tive candidate designs. The partitioning algorithm is looking at task/data allocation from a static analysis point of view to ensure that real-time computational requirements are met with a balanced load. The numberof entities that must be considered requires that parametric analysis interms of average or worst-case numbers be used in the partitioningprocess. The dynamic environment of the flight trainer computationaltask allocation requires the addition of network, queuing, and simula-tion (batch mode) tools to predict and assess the performance of a givenallocation partition with respect to representative scenario loads andresource management rules. The multiple-processor configuration con-tract is incorporating and expanding the conceptual repository toinclude the dynamic performance design aspects that are pertinent to

86

OIL,

r4

alternative computational candidate design evaluations for operationalflight trainers. The results of this related effort are to be publishedin the final report scheduled to be distributed on or about 31 Oct 80.

5.3 AREAS FOR FURTHER STUDY

Advancements in systems development and training features aresources of continuous change for flight trainer systems. A "good" systemtoday may be obsolete in 5 years or less if it does not possess modulardesign capabilities. This is particularly true of the computationalsystem, which must act as a coordinator, interface, and decision-makerto assist the human operators and commanders to better perform theirjobs. As new/upgraded flight trainer systems are required, the basicdesign models plus new/modified modules may very likely require reallo-cation of new processor, communication, and memory technologies. Twomajor areas of study have been isolated as the key to potential reali-zation of a truly automated software partitioning algorithm:

1. The employment and expansion of mathematical, mixed integer,program optimizer techniques for large-scale partitioningwith multiple objectives

2. The development of a master flight training simulator compu-tational subsystem design repository.

These two areas have been incorporated as major tasks associated withautomation of the partitioning algorithm described in Section 4.4.

In conclusion, automated software partitioning is feasible. Itwill require further study, design, and test steps that are directly re-lated to cdmputer facility selection for its implementation. The majortraining simulator candidate design impact would be toward standardiza-tion and separation of the software design representation and data fromprocessor hardware configuration representations and data. The resultsof the standardization would permit a consistent flight trainer computa-tional design automated repository to be established and used in both newdesign and current design evaluation tradeoffs in the areas of softwarepartitioning and predicted performance of multiple-processor configu-rations. The use of an optimizer will permit certain tradeoffs to beautomatically made and determined in a more straightforward manner, per-sitting more time for manual evaluation comparisons and decisions.

87I

APPENDIX A.

USER INPUTS

89 an g .'~ Raw

A.1 WORKSHEETS

I..

IL

29

Ii'44W

I" g Itb.JSLW

Ma mu Ma0 0

K4 4 4Ma MaW

t4II.

K,-.

4

Ma-a

- S -. - -

0

J J ]

4 1,-

I iiill!91

- A~k .*4.~

Ilit

. . . . . . . .

"~ . ...21 =.-. . .

ii J IJII]. j,

, , W

1 3' ]]JJ]1i11] 3i k

92

rc a z za

3 3 1

> J W .; 1

3 3C C C C 0 93

sj ji -0

> W

i~~ J- l in ---7

rg~1

MW ~ 1~

U"~ 191

oil

77V 11 11- -I -- -1 - - 11I~ ~ - - - -l - - - - - -

. . . . 3 . 3 . 3

K- I i11iii1iii

I.

- .J.95

14-]33J3J~

AD-AO9b 187 TELEDYNE BROWN ENGINEERING HUNTSVILLE ALA SYSTEMS Div F/B 9/2SOFTWARE PARTITI ON INO SCHEMES FOR ADVANCED SIMULAT ION COMPUTER -ETCIU)FEB Ri S .J CLYMER F33615-78 C 0013

UNCLASSIFIED AFHRL-TR-BO R2-PT- £ NL

I 11111.Illl"1 0 2-0li1111 I 1_ II I.

MICROCOPY RLS(CLU1ION I I SI CIIARI

U!

. . . . . . . . . . . . . . .

,. . . . .. .. . . ..

S2

iiJJJ]]]JJ]JJJJJ] IIJ J JJJ :I J J J J . . . . . . J

3 33333333333333 33

SI I :zi;

z lxl

.- . ,-,I", -' 4 4

3. ~ . ~ = - -

I-I- I * J96

7 v

. _ . _ .. . > 71

wI - i i I!

a ITS"-

1 3. 3 33 3 3 73 3 , ,

97- i-4.I

, . ( .w . . . .+ . . . .,

. . . . . .. . . -. . . .

- - - i IL.. .

-. -° - -. - - - -. ,- -

t3 3333 33 3 3333333

- - - -

" I , ,I 1 I

3 3i 3 3 3W 33 33 33 33-98

I

98

i'i JI1I1JJJ1JJ1JJJ-Ui-i 1JJJJJ1,1JJ.-

j ii II]IJJI]III iI ~~~I ' - I i ;i;,ii

!, 1! 11 1 1 11 ! 1

, J " I~ JJJJJJJJJJJJJJJJ-" " "l: "i , .i = ; :: i i i i I ' !99,'

- . ! .S.t . . .

I

..i . . ..i . . ..i . 1 .1

! i .j .j j-3 1

100

4 le 4c 4 4 49 49 le I

- - - - - -3 .4i i i . .i

iii 333111

101

p. - r - . - .-,

,. >

(j c W

I. Z .J4c54

0I0-

oil

10

A.2 DEFINITIONS

15 N

-0 7 . W at 1 0

Z p do 9 C 1 ai : so a.4 -'S. 0. -:o so 0 ,a. 0

hi VO .0~ J VdX 0~ 1Uec 00 UCC ID :iO f 2

: Co c p I xCW do04.J 2 S410 00 -i 046

4 C 0 d.0 11 p OmIL.0d a 31

-0 Vo CC SS @@ C) AD N* diio -o qc Mm

v Co ~ ~ a Lio 00141 a.0 au a-, 2 c a 0 9 aLS 0

L 0 a

di 0 A

W 04 0

.1103

Z 0 * 0.

Ja *1* v alpa

c4 063 .. 4

L 1-4 -0

Oa3, 100 U

U

Lh

c0

3 a3

uS

ca 0

J c 3 04 04 041L

4 s 4 31 cc Locem to 9 0lo a 1.6

V - --a- I-a V0 do ; N0 fl u

- - -a -.0 a a901

L a. A I- I- a- 9-6 0 0 U) 4 4

ha U0 0 0 0

Ink-~8 . ..

4 16 I

haa 0I

4 416

V .0

b3 0 0 9

)01 3 Q U

I. Z

2L k6 cL a - L 3PO i 0 -0 - 9 %. - 0 0 a

9, -. -s .9 0 s6No 61 b.1 z4 64 7

4~b SC ACSUS

L 0 0

2,

o i- . I- 050 S '

zZ N P0 3

P2 C C C4

11 Li

a a f a:at 5- 0

2 0 A.

0 V) U

0 u Ca a a C a

a a

0a a a -

adio

4 S m 106

2 I- S -'I

Z O 0 L a

hi sC 2 LU

r Ca. 0 05 LoILg~~a 09 .J.O

h* *Ua"

0 0 .1. -- A

cc not~ 00 em

L 0 o SL :O

a 6 1 di49 0 J o

'j L-*~

OW cL g-

107

x i *I

10 ..0 0 0 030

00 U S .S

~~ a.. 1 .3 .3413S

a 0)

.3d

Id~ 0a. U)

04 S S0 c

L. lo. a- A- a- 0

IqI

104

16

c 0 C

Z0 L4 1 SS

3 -a .

Jc 01

vi NI- 41 1SOL

~ *e-~4 I

Z u ~ II

Z0 X-c XC c n c g

02o 0 SSO

A10

z c 0

4 c c L 6 i

hi0 03

v c .0 aJhi 033

4 6 0 0 a033 0 3 N

0

00Uu

430

* 0hi 0

4 £

00

a v

L. 0

:~ 0aU30 *1 0

... .

LI c V - %

Z £3 C

a' 3- 3z a o &0 v 0 V *I v

a~ : .0

0B6

0 4A0 - * 4 L4

Im

ki IV, @ ia

I. L

A 5 tf

IO

v 4hi 6 6 6 S j

4 \0 \ \0 U .

* L L3 L3 do

~~ @0 0 06 c .

~ OS S 05 4' ' U B

I oxmx m - 3-C 6

S

,b?

J

bi E

z

J 0J 0

0 C

a a

L L

boo 0 be a

I- z a A.

16 0

L 0 .

L J. b N do

LS

im I

7w4

z .D v.4 C c

z 0 0 h

44 L a 0w L L. v

z40 0 9 £ CJc a a 0 0

- L, 15

I-0 8 0~

z a z

L)i

00

c 01 41 41 w

dO I0a .1 L.01

6 5 0 0 0b 0 ZU Z

114

4 LS 0 U a EW 0 00 0 A

Da 0 0 0 0 0

b.. I.-.u

u 0 1 1 0 0 . V. a 0 6

a 0 0

a; a 0 %.0 L I. 9 -

5,00 0

0 0 0 L

'o 0 01L

bo

ai

-r N1ca

doo a x LJ 2 LU 0 Cc 0 a.

4 ao c .i ai 53i .0 o . :m - 2 j

E~4 1~a C -46 C4

a -1SL L IL *j a ,i.1 0 2 3 0 *S A L aaa

a 3 C 40- 01 a A0

> L~41U c~ ~ -u *\ 3-@* S-N 3~ ~wS £ *S- -to

- ie E 41 g~ 4 05 16 I~5~L 6gB. - 60 gL

A a

>1 . 0 N* J41 a. 4

.3 U 3hi 3£

L ILw4

o a 0

: -i $a OW0 U 40

1 I. 0 .Sh i a i ahI

Is 0 so L

LUL ~ Sha e C

41*£116

II,

aco

L 02'a 0

hi0 0 0 ceI

0 0 0 4b. 16 b. b

S 0

Li 0- 9a

do a 00 L 0

0 SO L4- ~ L

: ~ 0 ca-be It .f3

L Ls

hi 117

L3 : I

to zs 3

hi NOL L~ L

L L i3 ! S0 0 6ce a 'f a- D L 0 Ai 2 '

at It at -t 3* aLSS( V* 2

\P. & 0 I v-S - K

UU

16 Ii 0 0 0

*a 39a .9ou no L 3 i L 8.

va b .C J Js a ai ft16 d" hi C mi0 *

hi NU* U 3 4)30 3.

- OWu N

I IL 1 6 I6 6 lb 6 I6 bi.

0 10. 3 1ii

a a -ad 16 -1- -- v8 .1 0 1

hi~~~~~ 0d dd b S 2 iL di dd a Lb A.2 L a3d L- 0I : K- a. -- "1 0 . ~

0 - K2 2 "AS a a- . o.*k a aa 2 5 to 0 Sc a aa0 L ao I. a*

hi S as as a a- - I3

I- L A. L S S S ga1L18

z a I S

L .4 0 - a0

ai 0i - a iu ia Ua J £ 0 -N

c 0 aD at di d - 0 doa 4 0 is A .DO 00 v 46

60~~~ L G C N E 1L6 CU U 10- vu C U6~ LCL~ -US.

U

b. L6 b.

*o di Si E U

£6 I. 3 I- S --

b. 0 *a .Do 0 i. h

u P- (140 1 u'

z L we 9o )6)6

Ei

4 hihi 0

a-

16

a 36

'0 e ! -o

IL-

120

z- C, CC C

z !C-a. - C 0, .

4 a d i .- a a N Nhi jD a C C & L0 I

uD a a 0

4~~ 16 a i- d.'. .5 -

0* o L I - u do a111 U-- H 33 3dm d

~.~ ; O Vi i 02iC 10 di"SO L 3 a .CLA

X0 ~ 0 60L* O 4 0 -

J 060 L 1-

0 IL ILI L ILo E

I L IL IL I

U~. 0 z

4A CC

M $- 0 L OW

a Zt a. o aahi 0 z a C

L CL i

0I v t 0Uk

hi ~~ ~ 1 41d i d

- S C UMEOW

Q C L

z 0 0E 0- A 3.3

CC LIS

L L 16 6

a- 4111 8.06£1t

0 S 0)

41 00 0 00Lc

do %.6 4 1

0) 0 06 0

U. Ozh 04 - O 4u

IM)

L4

-z - - -

• ",

0 6

ii L

- S

4 6 8 h

31 0 3. L

* Cu

A d -

3.OS)

LU~U

- S 8). 5.

I h 0

cUboi

.-

IS

hi123

do 41 41 0a L a a C V.0

c 0 10 o v 0 0 0II

hi ~L L A e4L ONZ C x DC0i .0 0 ! A

a a- O L VC~ a5 544 a 0 0 41 4 0 00 4" A I 00

*~~~~ 5L4- -1 £

a a~UL 00 3CDSO 300 - D

do a

41~ 4. u

3 00

aa

aIc u u U

K U U U

* 124

xA

13 N

op Cj *

hi 0 0 a4Ue d0 a 0

C 0 a 0

I.L £0 0 0al le as .

z ~ 9 ALCUC

IL)3 33 -

- 0

a-

IL4

Lh

'0 - Sreb34

4 1 0 so 1 03h41 -a 1.1 a a1* 13 0 c

1.3 0g 0 I*01

10 m

c1 01SO1-P4'6.J do 1.00 SI - 3 o q

In NC 2,-J E L U L U- U EM 3 U 3

J

b,

L L.

41 at p5

* i - l 41 4 9 0 G c .I- b1. 16-0

ha a L1-&&. 5

bo am

a~~ ~~ a ~

U ta ~

hi 0 U U~I UU 12UU

do *I

zI2 x

- L.-> -D & a

4IL441hhI U e £ if (

: S ~gg

.J z. 044 " L *

bi 9L W : ; --a-~ >a3a a*4 304 Ia~ c-au ugu Cu

a c* ha3 L

1 a. I-

'we 00 WO0266uS

L.2L a

6~ *5 3 127

a3 0L C

hi Ui "I

as - 0 ='"

hi t2 C .. .

- 0 Lu "

\p2 ,U --q I~ I

L L L 41 .0

'. z e a a2

: : o , : to,,

LSDCS UI "

.2 - 34 3S -S.,,5

hii4." L

5u u u u' ,5 ,u u

IL 3 c

& L4 f-.J U U 01 A LJ64 ~ ~ Av a.................. 5.9.....

£~~L N.SC M 9 ~ g u u0 0 OW

O~ J 0 ~ J J 1 t . J J

! ; P ZUuu I'DU UU UJ atI I- ~ -I- - I .- c

5 128

z

0 f.

LeL

U

16i

00

Z u

W L a. 0 : Lhi 413 c

L 0C 0

J Ci

.4

1 0 - Z a 0

I - P-U0ft 09 C

o n

E .J . 5129

o r 0S0 a a c C C

Z L.E C C 0 660 0 6 0 S

4 N 0 0 i o 0 di hiki .6 - S aC S

;- O 30 L m \2

Ja - L C 0 C aO C arL DA S8 0 0 0

c us a u. c I1- a u a 0 a 00z0

S., I. : : oc u3 c 9aS - C 0 0 C 0 0 C 00

SI- Jdi op Ud wh . i aZ hi Ud 0

0

In S

*. is 1- 61 S) 0

I

0 fya hi

hi 0 0 0 0 0w a. p. A a 0 31 a.0K go 11 0 1 - 1- .0 I- 03. a4

a 0a x a c I do a I

hi 0 c0 90 : :aa

r 6 c 1- a IV -

bo 0. C S Co C S 0 8 S 9

5 ** OLS56 1301

50 S IL04

la 41 1 4.10 S

L I.. L A

L @3 . 6

9. 21 35a4 0 L

I- aS do3 0541 id ~4I- 10 0a@

@3 -L fa soC 53 L-

~ *2 S.h. 131

c oW 0

9 L a E4 0 24 - a.

hiE - a 0 0 (3

Za~ 3d~ V ca

\ L a. *0 IL I

0 165 W a- L a

3~ Cu~ 3C* 3

~ EI Z - ra EU

E 1 (3 3 A

s 0 a J a 99 au x 0 - I0

s di di AiL 0 V C

L Eh

132

cS

090z L. C

hi u aa ugg a c ~

a I00 a .0 1& .0 v aChi- c~ 1 do 50

z o 0 0 : , *2 2 0C a 0, IV C

* ~ ~~~ 0 -cO 1

.4

W

-1 a z

s L, 0

hi S0 It 3o 0 4

.4 .4 0 a

0 hi

0 a 0 0K 0- Uc c 1- I - o,0

hi 0.0 1- w -u as cs

L - s-s

a133

u .8 0 A 3 A

A V - 0 a~L

.0 -D ;. a *u 63

\ a ~u-s s ue

g ~ I onL 6 40~a

~L ao aa 6 s 3

v 48* a be a

~L ~L~ L 3 002

* U) s -g -s -s8. ~134

hi U 43 8 8 430 srn 0 0

-c6Z LO 0 0 1. 0 .z.

.4 0- 1 41 .0 41o 41 a 0 0E~~~ L- V .I I .

la 0 0 0~ 0~ - a~ v U -

.Js s s : go go a -aa-> 6. 0 0 0 0 8 -8

* 3. 36 36 3 CE f1 451

01 41 0 0 0 0 0 S 0 0 0 0 I 3 .80 a u 03 03 In 0 31 .a A 0

go 66a0 654 *uw* 5

a1 )- 04 0 06 A ha D

w 413410 40 a h

z .0 c

u 17 9 r: - 9 L - - L-1

a c fl c 0 3 1. S

L .* S. S- S . . *C

L 3~ 3~ 3~ 3~

hat 4 . ( . . . .

Z El El E El l El135E

&whim

I, a, L ::L 6 L 2 L C

i I' a 21 a S 9 13I

2 6 83 8 0-

a Is ~ ~3. LI 6 L) C

2 ~ ~ I N ni £ 86 .*

0 ; IL IILiIL I - I

N $-

aL I-

) 9 - I L L L LUA

ii 39 0

16.

.461

fr

NIsz34 mlKma2-a

4*1 I.3 a.2 0

UmaS

S

03

0pg

3K

0 I..S pg 5 S

- mmI S 0~ B S S

pg g~ *.~ 6 * pg

hi ~ ~ S a- 0U**b* 3 ~ 4 ma maf m S a-

S ~ a. 51 4; ~ ~ 4ma S - a-\ SI. 4 ,- 0

3 U~@ mao use S -.S. - ml U2q ma a-a-ma U Oma maS

I- SRI- pg

U S - N fl 96 1U S pg pg pg - Spg

137

-V...

1~' S* -

APPENDIX B.

REPORT FORMATS

138 -Qu

B.1 USER ECHO REPORTS

Um

U1,

Pa- ",

rc

Ww x

X x(~ ( oX' x(~ (

x XxI xx x x

x ,' xw ( x ( w ( x

w- x( x(x X2 cW( xc x

CC x wS X x -

w (~

!w x( x( xw x( wC

-. ~ )( 140

(A 0%

I.- 0 b 031

2--

inCL -

C3Li)

.j I-

i -CP

m141

0% 0%t

O'(%i % m%LU m% at

Q: at

0% 0

u- % 0

IUJ

3-4 )P

I- CAi 30 11

wC 31-b

ago )PC 3.Cw

X 2C 330 1X( W 0C 0x X -CK 0

( be KW Kw -

8 KK -c .

3K w

LU .I- )O( 2K

142

0% Ch oTI TU Laj 2: o"o i

Ch- m~ O 0m m~ C~ a

ah m~ at

2:ILU LUI LU

s. ~ M O ON 0%

m~ ON 0%0% . .

Cs Fl.O

CL h C ChLU z ON m~ Ch

QOM%2 0% 0' Chtj~~ CA.~f 0'%~O

..- LUJ

o LU LU mh Oh

Oh% ON ONUCz LU Oo Cie 0%0 0 9

mU O ON Ohi

41O0 ON O

dc Ch C

Mh a

VCh LL U0 cPCh IE 0u

PC 1% zh 4h

x U&. I.I-

2-C 2: Ohjhah24I Z 4AUh a

PC -- Zw 0PCO Lat

atUTZ4a (nPC P

ui~P dC 0- Pc

143

xu x

C. >C x( w

CC4 U c 0- C

LA 9 i xuD 0D N< >C V

'4 CC) '

LLI-0 000 x w

W w

CD~ ~ -LuC )(x ~2c ii CL C

oA 41C~~C~uif . -

Lu Lu

Lu xC wC 2-C 0- 1-wc 3mc x

x w >-x NC 0C PK

w x

144u

I- ~ c A I

AnA

rm3.i- K DC N L

vc 3- W w vb* 3- 3c 1C3-

6--

3-P P C DC SK wC3-= C 3.C w PC P-C 3.c

S 51. PC P PC P PC 4

I~~ PC IC IC P C P2-C PCPC P PC C

3 .- m cI- c0

*- ~1 IC M1..- "C

Leiu~j

145

--S ;S

LL)

0-4

C'n 0:iiL5U

o La

Zn-I

t-

146

B.2 SOLUTION SUMMARIES

w ULL. U. L. U. W I. u. U. U. i

CA aU Q %a 0% at 0% %

m gi 0% 0% a$ %

4Aatg at 0% %0 at a%.

at at at % 0% 0% 0%

0% 1 at at0 0 % %

at0%. 0%. %

at0 0 %0%

*C w xc v 3C 3C x' PC

3' 3 '3 PC v CvDw v

w.2. w ~ w 31-w4

w ~L

I-.~* I- 0%0%

'k 3'

IC IPCa 3'cP

w3 -3' b 3'3C* ~ , ufi 3''33' 3* ~ ~ IC PC'33' 3* ~ ~ ~ ~ . I. ''3'' 33

PCa m Ile I NIA- ow 3'' '3m''331(3' Ic~' '3

* -314'

-al

S 0C 0% )K 0< 2K 3- 0% 0C:1-C 20C 0C 0% 0x 0% 240 0

Ix 3-% 0% 0% 0x 0C

ma. ~LL 0% 0 0 %.%0ii-. 0%- 0%. 0% 0 0 % .%

I ~ 0% 0% 0% 0% 0C0% 0% 0*L 0 *y M* 0%t 0% M% 0 % 0 % 0

0%~~a 0% 0 % 0D0 % 0

* 0% 0% 0 0% % 0% 0%c0

o 0% 0CA % 0% 0 % 0U-.

u PC

CPC24 x

o -~ a.. L. 148

* L LL mL. LL. U..-i tU- U-L A Z L A LA.L-

O- h Oa R~ m 7

o ON M~ 0% M~ 0%

.= Ll OX X~ 0. X. X

=) (A m 0% 0% CAC.N0 0. 0. 0. m.

I- 0. ~ 0. x~ 0

LaJ CA ON ON 0% M.U-

V) 4m (71 0. 0% 4mwL ON 0. C7% 0. 0.w m. a% 0. m. Mo o c. 0 0. 0. M.

C- M. 0. i CA 0.

tt

S x X x x4 >I.-.

x >4c x .: l

oC V>~ C >C >C 3K u..,_U )( >C ( : x

u >C X X >lc

Co x 0C x 0x Xccx0 >

ai Ot a a

0..0

uj 0. 0. 0.atIn cW( x x

X >IC 3K

149

I .~ L.. U.. LL.... LL U.. LL- UA.

I U.

w hm 0

4m a% a% 4m

I--

.- E ONa" a a% al~

V)nc. al al a% 0%

o 0

U, al o al ON_j, a% ON ON 0M

w = u 0% ON~ O 0% W

40 a4 cc~ 0m O ON 4m Ca..

0

V) 0! 0 0! C!V)4 M ) ON C

Ii ONC.

ON XE, C) ON0 0 .

oi 0%O ha

00 >

cc 0

IjJ w' " ~ 0

Wc 0 ' ~o

ISO

.... LL. LL. IL. U-. LL. U.U-

~J 0) 0) 0) 0% 0) 0%E ) 0% 0% 4m at 0%- ) 0% 0% 0t 0% 0%

o 0% 0% 0% 0m 0) 0%

0m 0) 0) 0 0) 0%at0)

ui 0% 0% 0% 0) 0) 0%

u.i 0) 0) 0) 0) 0% 0%

~ 0) 0 0) 0 0)C0

o C 0) 0w 0) 0 u0

oD CD >C C )< w( w R-i. x( 3K 2-< -C

-( :0 C >C >

dw 0) 0) 0)

U-

2c CA 0) 07$=i 0) 0) atLU 0) 0) 0)

x w)IC 3-C(

be~ ~

z (~

DA138