INTRODUCTIONTO OPTIMUM

DESIGN

THIRD EDITIONJASBIR S. ARORAThe University of Iowa

College of Engineering

Iowa City, Iowa

AMSTERDAM• BOSTON • HEIDELBERG • LONDON

NEW YORK • OXFORD • PARIS • SAN DIEGO

SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO

Academic Press is an imprint of Elsevier

Academic Press is an imprint of Elsevier225 Wyman Street, Waltham, MA 02451, USAThe Boulevard, Langford Lane, Kidlington, Oxford, OX5 1GB, UK

© 2012 Elsevier Inc. All rights reserved

No part of this publication may be reproduced or transmitted in any form or by any means, electronic ormechanical, including photocopying, recording, or any information storage and retrieval system, withoutpermission in writing from the publisher. Details on how to seek permission, further information about thePublisher’s permissions policies and our arrangements with organizations such as the Copyright ClearanceCenter and the Copyright Licensing Agency,can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher(other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden ourunderstanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating andusing any information, methods, compounds, or experiments described herein.In using such information or methods they should be mindful of their own safety and the safetyof others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume anyliability for any injury and/or damage to persons or property as a matter of products liability, negligence orotherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the materialherein.

MATLABs is a trademark of TheMathWorks, Inc., and is used with permission. TheMathWorks does notwarrant the accuracy of the text or exercises in this book. This book’s use or discussion of the MATLABs

software or related products does not constitute endorsement or sponsorship by TheMathWorks of a particularpedagogical approach or particular use of the MATLABs software.

MATLABs and Handle Graphicss are registered trademarks of TheMathWorks, Inc.

Library of Congress Cataloging-in-Publication DataArora, Jasbir S.

Introduction to optimum design / Jasbir Arora. � 3rd ed.p. cm.

Includes bibliographical references and index.ISBN 978-0-12-381375-6 (hardback)

1. Engineering design�Mathematical models. I. Title.TA174.A76 2011620’.0042015118�dc23 2011026976

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

For information on all Academic Press publicationsvisit our Web site at www.elsevierdirect.com

Printed in the United States

11 12 13 14 15 10 9 8 7 6 5 4 3 2 1

ToRuheeRitaand

in memory of my parentsBalwant KaurWazir Singh

Contents

Preface to Third Edition xiii

Acknowledgments xv

Key Symbols and Abbreviations xvi

ITHE BASIC CONCEPTS

1 Introduction to Design Optimization 1

1.1 The Design Process 21.2 Engineering Design versus Engineering

Analysis 41.3 Conventional versus Optimum Design

Process 41.4 Optimum Design versus Optimal Control 61.5 Basic Terminology and Notation 6

1.5.1 Points and Sets 61.5.2 Notation for Constraints 81.5.3 Superscripts/Subscripts and Summation

Notation 91.5.4 Norm/Length of a Vector 101.5.5 Functions 111.5.6 Derivatives of Functions 121.5.7 U.S.�British versus SI Units 13

2 Optimum Design Problem

Formulation 17

2.1 The Problem Formulation Process 182.1.1 Step 1: Project/Problem Description 182.1.2 Step 2: Data and Information

Collection 192.1.3 Step 3: Definition of Design Variables 202.1.4 Step 4: Optimization Criterion 212.1.5 Step 5: Formulation of Constraints 22

2.2 Design of a Can 252.3 Insulated Spherical Tank Design 26

2.4 Sawmill Operation 282.5 Design of a Two-Bar Bracket 302.6 Design of a Cabinet 37

2.6.1 Formulation 1 for Cabinet Design 372.6.2 Formulation 2 for Cabinet Design 382.6.3 Formulation 3 for Cabinet Design 39

2.7 Minimum-Weight Tubular Column Design 402.7.1 Formulation 1 for Column Design 412.7.2 Formulation 2 for Column Design 41

2.8 Minimum-Cost Cylindrical Tank Design 422.9 Design of Coil Springs 432.10 Minimum-Weight Design of a Symmetric

Three-Bar Truss 462.11 A General Mathematical Model for Optimum

Design 502.11.1 Standard Design Optimization

Model 502.11.2 Maximization Problem Treatment 512.11.3 Treatment of "Greater Than Type"

Constraints 512.11.4 Application to Different Engineering

Fields 522.11.5 Important Observations about the

Standard Model 522.11.6 Feasible Set 532.11.7 Active/Inactive/Violated

Constraints 532.11.8 Discrete and Integer Design

Variables 542.11.9 Types of Optimization Problems 55

Exercises for Chapter 2 56

3 Graphical Optimization and Basic

Concepts 65

3.1 Graphical Solution Process 653.1.1 Profit Maximization Problem 653.1.2 Step-by-Step Graphical Solution

Procedure 67

v

3.2 Use of Mathematica for GraphicalOptimization 713.2.1 Plotting Functions 723.2.2 Identification and Shading of Infeasible

Region for an Inequality 733.2.3 Identification of Feasible Region 733.2.4 Plotting of Objective Function

Contours 743.2.5 Identification of Optimum Solution 74

3.3 Use of MATLAB for Graphical Optimization 753.3.1 Plotting of Function Contours 753.3.2 Editing of Graph 77

3.4 Design Problem with Multiple Solutions 773.5 Problem with Unbounded Solution 793.6 Infeasible Problem 793.7 Graphical Solution for the Minimum-Weight

Tubular Column 803.8 Graphical Solution for a Beam Design

Problem 82Exercises for Chapter 3 83

4 Optimum Design Concepts: Optimality

Conditions 95

4.1 Definitions of Global and Local Minima 964.1.1 Minimum 974.1.2 Existence of a Minimum 102

4.2 Review of Some Basic Calculus Concepts 1034.2.1 Gradient Vector: Partial Derivatives

of a Function 1034.2.2 Hessian Matrix: Second-Order Partial

Derivatives 1054.2.3 Taylor’s Expansion 1064.2.4 Quadratic Forms and Definite

Matrices 1094.3 Concept of Necessary and Sufficient

Conditions 1154.4 Optimality Conditions: Unconstrained

Problem 1164.4.1 Concepts Related to Optimality

Conditions 1164.4.2 Optimality Conditions for Functions

of a Single Variable 1174.4.3 Optimality Conditions for Functions

of Several Variables 1224.5 Necessary Conditions: Equality-Constrained

Problem 1304.5.1 Lagrange Multipliers 131

4.5.2 Lagrange Multiplier Theorem 1354.6 Necessary Conditions for a General Constrained

Problem 1374.6.1 The Role of Inequalities 1374.6.2 Karush-Kuhn-Tucker Necessary

Conditions 1394.6.3 Summary of the KKT Solution

Approach 1524.7 Postoptimality Analysis: The Physical Meaning

of Lagrange Multipliers 1534.7.1 Effect of Changing Constraint

Limits 1534.7.2 Effect of Cost Function Scaling on the

Lagrange Multipliers 1564.7.3 Effect of Scaling a Constraint on Its

Lagrange Multiplier 1584.7.4 Generalization of Constraint Variation

Sensitivity Result 1594.8 Global Optimality 159

4.8.1 Convex Sets 1604.8.2 Convex Functions 1624.8.3 Convex Programming Problem 1644.8.4 Transformation of a Constraint 1684.8.5 Sufficient Conditions for Convex

Programming Problems 1694.9 Engineering Design Examples 171

4.9.1 Design of a Wall Bracket 1714.9.2 Design of a Rectangular

Beam 174Exercises for Chapter 4 178

5 More on Optimum Design Concepts:

Optimality Conditions 189

5.1 Alternate Form of KKT NecessaryConditions 189

5.2 Irregular Points 1925.3 Second-Order Conditions for Constrained

Optimization 1945.4 Second-Order Conditions for Rectangular

Beam Design Problem 1995.5 Duality in Nonlinear Programming 201

5.5.1 Local Duality: Equality ConstraintsCase 201

5.5.2 Local Duality: The Inequality ConstraintsCase 206

Exercises for Chapter 5 208

vi CONTENTS

IINUMERICAL METHODS FORCONTINUOUS VARIABLE

OPTIMIZATION

6 Optimum Design with Excel

Solver 213

6.1 Introduction to Numerical Methods forOptimum Design 2136.1.1 Classification of Search Methods 2146.1.2 What to Do If the Solution Process

Fails 2156.1.3 Simple Scaling of Variables 217

6.2 Excel Solver: An Introduction 2186.2.1 Excel Solver 2186.2.2 Roots of a Nonlinear Equation 2196.2.3 Roots of a Set of Nonlinear

Equations 2226.3 Excel Solver for Unconstrained Optimization

Problems 2246.4 Excel Solver for Linear Programming

Problems 2256.5 Excel Solver for Nonlinear Programming:

Optimum Design of Springs 2276.6 Optimum Design of Plate Girders Using Excel

Solver 2316.7 Optimum Design of Tension Members 2386.8 Optimum Design of Compression Members 243

6.8.1 Formulation of the Problem 2436.8.2 Formulation of the Problem for Inelastic

Buckling 2476.8.3 Formulation of the Problem for Elastic

Buckling 2496.9 Optimum Design of Members for Flexure 2506.10 Optimum Design of Telecommunication

Poles 263Exercises for Chapter 6 271

7 Optimum Design with MATLAB 275

7.1 Introduction to the Optimization Toolbox 2757.1.1 Variables and Expressions 2757.1.2 Scalar, Array, and Matrix Operations 2767.1.3 Optimization Toolbox 276

7.2 Unconstrained Optimum Design Problems 2787.3 Constrained Optimum Design Problems 2817.4 Optimum Design Examples with MATLAB 284

7.4.1 Location of Maximum Shear Stress forTwo Spherical Bodies in Contact 284

7.4.2 Column Design for Minimum Mass 2867.4.3 Flywheel Design for Minimum Mass 290

Exercises for Chapter 7 294

8 Linear Programming Methods

for Optimum Design 299

8.1 Linear Functions 3008.2 Definition of a Standard Linear Programming

Problem 3008.2.1 Standard LP Definition 3008.2.2 Transcription to Standard LP 302

8.3 Basic Concepts Related to Linear ProgrammingProblems 3058.3.1 Basic Concepts 3058.3.2 LP Terminology 3108.3.3 Optimum Solution to LP Problems 313

8.4 Calculation of Basic Solutions 3148.4.1 The Tableau 3148.4.2 The Pivot Step 3168.4.3 Basic Solutions to Ax5 b 317

8.5 The Simplex Method 3218.5.1 The Simplex 3218.5.2 Basic Steps in the Simplex

Method 3218.5.3 Basic Theorems of Linear

Programming 3268.6 The Two-Phase Simplex Method—Artificial

Variables 3348.6.1 Artificial Variables 3348.6.2 Artificial Cost Function 3368.6.3 Definition of the Phase I Problem 3368.6.4 Phase I Algorithm 3378.6.5 Phase II Algorithm 3398.6.6 Degenerate Basic Feasible Solution 345

8.7 Postoptimality Analysis 3488.7.1 Changes in Constraint Limits 3488.7.2 Ranging Right-Side Parameters 3548.7.3 Ranging Cost Coefficients 3598.7.4 Changes in the Coefficient

Matrix 361Exercises for Chapter 8 363

viiCONTENTS

9 More on Linear Programming Methods

for Optimum Design 377

9.1 Derivation of the Simplex Method 3779.1.1 General Solution to Ax5 b 3779.1.2 Selection of a Nonbasic Variable that

Should Become Basic 3799.1.3 Selection of a Basic Variable that Should

Become Nonbasic 3819.1.4 Artificial Cost Function 3829.1.5 The Pivot Step 3849.1.6 Simplex Algorithm 384

9.2 An Alternate Simplex Method 3859.3 Duality in Linear Programming 387

9.3.1 Standard Primal LP Problem 3879.3.2 Dual LP Problem 3889.3.3 Treatment of Equality Constraints 3899.3.4 Alternate Treatment of Equality

Constraints 3919.3.5 Determination of the Primal Solution

from the Dual Solution 3929.3.6 Use of the Dual Tableau to Recover

the Primal Solution 3959.3.7 Dual Variables as Lagrange

Multipliers 3989.4 KKT Conditions for the LP Problem 400

9.4.1 KKT Optimality Conditions 4009.4.2 Solution to the KKT Conditions 400

9.5 Quadratic Programming Problems 4029.5.1 Definition of a QP Problem 4029.5.2 KKT Necessary Conditions for the QP

Problem 4039.5.3 Transformation of KKT Conditions 4049.5.4 The Simplex Method for Solving QP

Problem 405Exercises for Chapter 9 409

10 Numerical Methods for Unconstrained

Optimum Design 411

10.1 Gradient-Based and Direct SearchMethods 411

10.2 General Concepts: Gradient-BasedMethods 41210.2.1 General Concepts 41310.2.2 A General Iterative Algorithm 413

10.3 Descent Direction and Convergence ofAlgorithms 41510.3.1 Descent Direction and Descent

Step 415

10.3.2 Convergence of Algorithms 41710.3.3 Rate of Convergence 417

10.4 Step Size Determination: Basic Ideas 41810.4.1 Definition of the Step Size

Determination Subproblem 41810.4.2 Analytical Method to Compute Step

Size 41910.5 Numerical Methods to Compute Step Size 421

10.5.1 General Concepts 42110.5.2 Equal-Interval Search 42310.5.3 Alternate Equal-Interval Search 42510.5.4 Golden Section Search 425

10.6 Search Direction Determination: TheSteepest-Descent Method 431

10.7 Search Direction Determination: TheConjugate Gradient Method 434

10.8 Other Conjugate Gradient Methods 437Exercises for Chapter 10 438

11 More on Numerical Methods for

Unconstrained Optimum Design 443

11.1 More on Step Size Determination 44411.1.1 Polynomial Interpolation 44411.1.2 Inexact Line Search: Armijo’s

Rule 44811.1.3 Inexact Line Search: Wolfe

Conditions 44911.1.4 Inexact Line Search: Goldstein Test 450

11.2 More on the Steepest-Descent Method 45111.2.1 Properties of the Gradient Vector 45111.2.2 Orthogonality of Steepest-Descent

Directions 45411.3 Scaling of Design Variables 45611.4 Search Direction Determination: Newton’s

Method 45911.4.1 Classical Newton’s Method 46011.4.2 Modified Newton’s Method 46111.4.3 Marquardt Modification 465

11.5 Search Direction Determination: Quasi-NewtonMethods 46611.5.1 Inverse Hessian Updating: The DFP

Method 46711.5.2 Direct Hessian Updating: The BFGS

Method 47011.6 Engineering Applications of Unconstrained

Methods 47211.6.1 Data Interpolation 47211.6.2 Minimization of Total Potential

Energy 473

viii CONTENTS

11.6.3 Solutions of Nonlinear Equations 47511.7 Solutions to Constrained Problems Using

Unconstrained Optimization Methods 47711.7.1 Sequential Unconstrained Minimization

Techniques 47811.7.2 Augmented Lagrangian (Multiplier)

Methods 47911.8 Rate of Convergence of Algorithms 481

11.8.1 Definitions 48111.8.2 Steepest-Descent Method 48211.8.3 Newton’s Method 48311.8.4 Conjugate Gradient Method 48411.8.5 Quasi-Newton Methods 484

11.9 Direct Search Methods 48511.9.1 Univariate Search 48511.9.2 Hooke-Jeeves Method 486

Exercises for Chapter 11 487

12 Numerical Methods for Constrained

Optimum Design 491

12.1 Basic Concepts Related to NumericalMethods 49212.1.1 Basic Concepts Related to Algorithms

for Constrained Problems 49212.1.2 Constraint Status at a Design

Point 49512.1.3 Constraint Normalization 49612.1.4 The Descent Function 49812.1.5 Convergence of an Algorithm 498

12.2 Linearization of the Constrained Problem 49912.3 The Sequential Linear Programming

Algorithm 50612.3.1 Move Limits in SLP 50612.3.2 An SLP Algorithm 50812.3.3 The SLP Algorithm: Some

Observations 51212.4 Sequential Quadratic Programming 51312.5 Search Direction Calculation: The QP

Subproblem 51412.5.1 Definition of the QP Subproblem 51412.5.2 Solving of the QP Subproblem 518

12.6 The Step Size Calculation Subproblem 52012.6.1 The Descent Function 52012.6.2 Step Size Calculation: Line

Search 52212.7 The Constrained Steepest-Descent

Method 52512.7.1 The CSD Algorithm 526

12.7.2 The CSD Algorithm: SomeObservations 527

Exercises for Chapter 12 527

13 More on Numerical Methods

for Constrained Optimum

Design 533

13.1 Potential Constraint Strategy 53413.2 Inexact Step Size Calculation 537

13.2.1 Basic Concept 53713.2.2 Descent Condition 53813.2.3 CSD Algorithm with Inexact Step

Size 54213.3 Bound-Constrained Optimization 549

13.3.1 Optimality Conditions 54913.3.2 Projection Methods 55013.3.3 Step Size Calculation 552

13.4 Sequential Quadratic Programming: SQPMethods 55313.4.1 Derivation of the Quadratic

Programming Subproblem 55413.4.2 Quasi-Newton Hessian

Approximation 55713.4.3 SQP Algorithm 55813.4.4 Observations on SQP Methods 56113.4.5 Descent Functions 563

13.5 Other Numerical Optimization Methods 56413.5.1 Method of Feasible Directions 56413.5.2 Gradient Projection Method 56613.5.3 Generalized Reduced Gradient

Method 56713.6 Solution to the Quadratic Programming

Subproblem 56913.6.1 Solving the KKT Necessary

Conditions 57013.6.2 Direct Solution to the QP

Subproblem 571Exercises for Chapter 13 572

14 Practical Applications

of Optimization 575

14.1 Formulation of Practical Design OptimizationProblems 57614.1.1 General Guidelines 57614.1.2 Example of a Practical Design

Optimization Problem 577

ixCONTENTS

14.2 Gradient Evaluation of Implicit Functions 58214.3 Issues in Practical Design Optimization 587

14.3.1 Selection of an Algorithm 58714.3.2 Attributes of a Good Optimization

Algorithm 58814.4 Use of General-Purpose Software 589

14.4.1 Software Selection 58914.4.2 Integration of an Application into

General-Purpose Software 58914.5 Optimum Design of Two-Member Frame with

Out-of-Plane Loads 59014.6 Optimum Design of a Three-Bar Structure for

Multiple Performance Requirements 59214.6.1 Symmetric Three-Bar Structure 59214.6.2 Asymmetric Three-Bar Structure 59414.6.3 Comparison of Solutions 598

14.7 Optimal Control of Systems by NonlinearProgramming 59814.7.1 A Prototype Optimal Control

Problem 59814.7.2 Minimization of Error in State

Variable 60214.7.3 Minimum Control Effort Problem 60814.7.4 Minimum Time Control Problem 60914.7.5 Comparison of Three Formulations

for the Optimal Control of SystemMotion 611

14.8 Alternative Formulations for StructuralOptimization Problems 612

14.9 Alternative Formulations for Time-DependentProblems 613

Exercises for Chapter 14 615

IIIADVANCED AND MODERN

TOPICS ON OPTIMUMDESIGN

15 Discrete Variable Optimum Design

Concepts and Methods 619

15.1 Basic Concepts and Definitions 62015.1.1 Definition of Mixed Variable Optimum

Design Problem: MV-OPT 62015.1.2 Classification of Mixed Variable

Optimum Design Problems 621

15.1.3 Overview of Solution Concepts 62215.2 Branch-and-Bound Methods 623

15.2.1 Basic BBM 62315.2.2 BBM with Local Minimization 62515.2.3 BBM for General MV-OPT 627

15.3 Integer Programming 62815.4 Sequential Linearization Methods 62915.5 Simulated Annealing 63015.6 Dynamic Rounding-Off Method 63215.7 Neighborhood Search Method 63315.8 Methods for Linked Discrete Variables 63315.9 Selection of a Method 63515.10 Adaptive Numerical Method for Discrete

Variable Optimization 63615.10.1 Continuous Variable

Optimization 63615.10.2 Discrete Variable Optimization 637

Exercises for Chapter 15 639

16 Genetic Algorithms for Optimum

Design 643

16.1 Basic Concepts and Definitions 64416.2 Fundamentals of Genetic Algorithms 64616.3 Genetic Algorithm for Sequencing-Type

Problems 65116.4 Applications 653Exercises for Chapter 16 653

17 Multi-objective Optimum Design

Concepts and Methods 657

17.1 Problem Definition 65817.2 Terminology and Basic Concepts 660

17.2.1 Criterion Space and Design Space 66017.2.2 Solution Concepts 66217.2.3 Preferences and Utility Functions 66517.2.4 Vector Methods and Scalarization

Methods 66617.2.5 Generation of Pareto Optimal Set 66617.2.6 Normalization of Objective

Functions 66717.2.7 Optimization Engine 667

17.3 Multi-objective Genetic Algorithms 66717.4 Weighted Sum Method 67117.5 Weighted Min-Max Method 67217.6 Weighted Global Criterion Method 67317.7 Lexicographic Method 674

x CONTENTS

17.8 Bounded Objective Function Method 67517.9 Goal Programming 67617.10 Selection of Methods 677Exercises for Chapter 17 678

18 Global Optimization Concepts

and Methods 681

18.1 Basic Concepts of Solution Methods 68218.1.1 Basic Solution Concepts 68218.1.2 Overview of Methods 683

18.2 Overview of Deterministic Methods 68418.2.1 Covering Methods 68418.2.2 Zooming Method 68518.2.3 Methods of Generalized Descent 68618.2.4 Tunneling Method 688

18.3 Overview of Stochastic Methods 68918.3.1 Pure Random Search Method 69018.3.2 Multistart Method 69118.3.3 Clustering Methods 69118.3.4 Controlled Random Search: Nelder-

Mead Method 69418.3.5 Acceptance-Rejection Methods 69718.3.6 Stochastic Integration 698

18.4 Two Local-Global Stochastic Methods 69918.4.1 Conceptual Local-Global

Algorithm 69918.4.2 Domain Elimination Method 70018.4.3 Stochastic Zooming Method 70218.4.4 Operations Analysis of Methods 702

18.5 Numerical Performance of Methods 70518.5.1 Summary of Features of Methods 70518.5.2 Performance of Some Methods with

Unconstrained Problems 70618.5.3 Performance of Stochastic Zooming and

Domain Elimination Methods 70718.5.4 Global Optimization of Structural

Design Problems 708Exercises for Chapter 18 710

19 Nature-Inspired Search

Methods 713

19.1 Differential Evolution Algorithm 71419.1.1 Generation of an Initial

Population 71519.1.2 Generation of a Donor Design 716

19.1.3 Crossover Operation to Generatethe Trial Design 716

19.1.4 Acceptance/Rejection of the TrialDesign 717

19.1.5 DE Algorithm 71719.2 Ant Colony Optimization 718

19.2.1 Ant Behavior 71819.2.2 ACO Algorithm for the Traveling

Salesman Problem 72119.2.3 ACO Algorithm for Design

Optimization 72419.3 Particle Swarm Optimization 727

19.3.1 Swarm Behavior and Terminology 72719.3.2 Particle Swarm Optimization

Algorithm 728Exercises for Chapter 19 729

20 Additional Topics on Optimum

Design 731

20.1 Meta-Models for Design Optimization 73120.1.1 Meta-Model 73120.1.2 Response Surface Method 73320.1.3 Normalization of Variables 737

20.2 Design of Experiments for Response SurfaceGeneration 741

20.3 Discrete Design with Orthogonal Arrays 74920.4 Robust Design Approach 754

20.4.1 Robust Optimization 75420.4.2 The Taguchi Method 761

20.5 Reliability-Based Design Optimization—Designunder Uncertainty 76720.5.1 Review of Background Material for

RBDO 76820.5.2 Calculation of the Reliability

Index 77420.5.3 Formulation of Reliability-Based Design

Optimization 784

Appendix A: Vector and Matrix

Algebra 785

A.1 Definition of Matrices 785A.2 Types of Matrices and Their Operations 787

A.2.1 Null Matrix 787A.2.2 Vector 787A.2.3 Addition of Matrices 787

xiCONTENTS

A.2.4 Multiplication of Matrices 788A.2.5 Transpose of a Matrix 790A.2.6 Elementary Row�Column

Operations 790A.2.7 Equivalence of Matrices 790A.2.8 Scalar Product—Dot Product of

Vectors 790A.2.9 Square Matrices 791A.2.10 Partitioning of Matrices 791

A.3 Solving n Linear Equations in nUnknowns 792A.3.1 Linear Systems 792A.3.2 Determinants 793A.3.3 Gaussian Elimination Procedure 796A.3.4 Inverse of a Matrix: Gauss-Jordan

Elimination 800A.4 Solution to m Linear Equations in n

Unknowns 803A.4.1 Rank of a Matrix 803A.4.2 General Solution of m3 n Linear

Equations 804A.5 Concepts Related to a Set of Vectors 810

A.5.1 Linear Independence of a Set ofVectors 810

A.5.2 Vector Spaces 814A.6 Eigenvalues and Eigenvectors 816A.7 Norm and Condition Number of a Matrix 818

A.7.1 Norm of Vectors and Matrices 818A.7.2 Condition Number of a Matrix 819

Exercises for Appendix A 819

Appendix B: Sample Computer

Programs 823

B.1 Equal Interval Search 823B.2 Golden Section Search 826B.3 Steepest-Descent Method 829B.4 Modified Newton’s Method 829

Bibliography 841

Answers to Selected Exercises 851

Index 861

xii CONTENTS

Preface to Third Edition

The philosophy of this third edition ofIntroduction to Optimum Design is to providereaders with an organized approach toengineering design optimization that isboth rigorous and simple, that illustratesbasic concepts and procedures with simpleexamples, and that demonstrates the appli-cability of these concepts and procedures toengineering design problems. The key stepin the optimum design process is the for-mulation of a design problem as an optimi-zation problem, which is emphasized andillustrated with examples. In addition, in-sights into, and interpretations of, optimal-ity conditions are discussed and illustrated.

Two main objectives were set for thethird edition: (1) to enhance the presenta-tion of the book’s content and (2) to includeadvanced topics so that the book will besuitable for higher-level courses on designoptimization. The first objective is achievedby making the material more concise, orga-nizing it with more second-, third-, andfourth-level headings, and using illustra-tions in example problems that have moredetails. The second objective is achieved byincluding several new topics suitable forboth alternate basic and advanced courses.

New topics include duality in nonlinearprogramming, optimality conditions for theSimplex method, the rate of convergence ofiterative algorithms, solution methods forquadratic programming problems, directsearch methods, nature-inspired searchmethods, response surface methods, designof experiments, robust design optimization,and reliability-based design optimization.

This edition can be broadly divided intothree parts. Part I, Chapters 1 through 5, pre-sents the basic concepts related to optimumdesign and optimality conditions. Part II,Chapters 6 through 14, treats numericalmethods for continuous variable optimiza-tion problems and their applications. Finally,Part III, Chapters 15 through 20, offersadvanced and modern topics on optimumdesign, including methods that do notrequire derivatives of the problem functions.

Introduction to Optimum Design, ThirdEdition, can be used to construct severaltypes of courses depending on the instruc-tor’s preference and learning objectives forstudents. Three course types are suggested,although several variations are possible.

Undergraduate/First-Year GraduateCourse

Topics for an undergraduate and/or first-year graduate course include

• Formulation of optimization problems(Chapters 1 and 2)

• Optimization concepts using thegraphical method (Chapter 3)

• Optimality conditions for unconstrainedand constrained problems (Chapter 4)

• Use of Excel and MATLABs illustratingoptimum design of practical problems(Chapters 6 and 7)

• Linear programming (Chapter 8)• Numerical methods for unconstrained

and constrained problems (Chapters 10and 12)

xiii

The use of Excel and MATLAB is to beintroduced mid-semester so that studentshave a chance to formulate and solve morechallenging project-type problems by seme-ster’s end. Note that advanced project-typeexercises and sections with advanced mate-rial are marked with an asterisk (*) next tosection headings, which means that theymay be omitted for this course.

First Graduate-Level Course

Topics for a first graduate-level courseinclude

• Theory and numerical methods forunconstrained optimization (Chapters 1through 4 and 10 and 11)

• Theory and numerical methods forconstrained optimization (Chapters 4, 5,12, and 13)

• Linear and quadratic programming(Chapters 8 and 9)

The pace of material coverage should befaster for this course type. Students cancode some of the algorithms into computerprograms and solve practical problems.

Second Graduate-Level Course

This course presents advanced topics onoptimum design:

• Duality theory in nonlinearprogramming, rate of convergence ofiterative algorithms, derivation ofnumerical methods, and direct searchmethods (Chapters 1 through 14)

• Methods for discrete variable problems(Chapter 15)

• Nature-inspired search methods(Chapters 16 and 19)

• Multi-objective optimization(Chapter 17)

• Global optimization (Chapter 18)• Response surface methods, robust

design, and reliability-based designoptimization (Chapter 20)

During this course, students write com-puter programs to implement some of thenumerical methods and to solve practicalproblems.

xiv PREFACE TO THIRD EDITION

Acknowledgments

I would like to give special thanks to mycolleague, Professor Karim Abdel-Malek,Director of the Center for Computer-AidedDesign at The University of Iowa, for hisenthuastic support for this project and forgetting me involved with the very excitingresearch taking place in the area of digitalhuman modeling under the Virtual SoldierResearch Program.

I would also like to acknowledge the con-tributions of the following colleagues:Professor Tae Hee Lee provided me with afirst draft of the material for Chapter 7;Dr. Tim Marler provided me with a firstdraft of the material for Chapter 17; Profes-sor G. J. Park provided me with a first draftof the material for Chapter 20; and Drs.Marcelo A. da Silva and Qian Wang pro-vided me with a first draft of some of thematerial for Chapter 6. Their contributionswere invaluable in the polishing of thesechapters. In addition, Dr. Tim Marler, Dr.Yujiang Xiang, Dr. Rajan Bhatt, Dr. HyunJoon Chung, and John Nicholson providedme with valuable input for improving thepresentation of material in some chapters. Iwould also like to acknowledge the help ofJun Choi, Hyun-Jung Kwon, and JohnNicholson with parts of the book’s solutionsmanual.

I am grateful to numerous colleagues andfriends around the globe for their fruitfulassociations with me and for discussions on

the subject of optimum design. I appreciatemy colleagues at The University of Iowawho used the previous editions of the bookto teach an undergraduate course on opti-mum design: Professors Karim Abdel-Malek, Asghar Bhatti, Kyung Choi, VijayGoel, Ray Han, Harry Kane, George Lance,and Emad Tanbour. Their input and sugges-tions greatly helped me improve the presen-tation of material in the first 12 chapters ofthis edition. I would also like to acknowl-edge all of my former graduate studentswhose thesis work on various topics of opti-mization contributed to the broadening ofmy horizons on the subject.

I would like to thank Bob Canfield,Hamid Torab, Jingang Yi, and others forreviewing various parts the third edition.Their suggestions helped me greatly in itsfine-tuning. I would also like to thank SteveMerken and Marilyn Rash at Elsevier fortheir superb handling of the manuscriptand production of the book. I also thankMelanie Laverman for help with the editingof some of the book’s chapters.

I am grateful to the Department of Civiland Environmental Engineering, Center forComputer-Aided Design, College of Engi-neering, and The University of Iowa forproviding me with time, resources, andsupport for this very satisfying endeavor.Finally, I would like to thank my familyand friends for their love and support.

xv

Key Symbols and Abbreviations

(a �b) Dot product of vectors a and b;aTb

c(x) Gradient of cost function, rf(x)f(x) Cost function to be minimizedgj(x) jth inequality constrainthi(x) ith equality constraintm Number of inequality

constraintsn Number of design variablesp Number of equality constraintsx Design variable vector of

dimension nxi ith component of design vari-

able vector xx(k) kth design variable vector

Note: A superscript (i) indicates optimumvalue for a variable, (ii) indicates advancedmaterial section, and (iii) indicates a project-type exercise.

ACO Ant colony optimizationBBM Branch-and-bound methodCDF Cumulative distribution

functionCSD Constrained steepest descentDE Differential evolution; Domain

eliminationGA Genetic algorithmILP Integer linear programmingKKT Karush-Kuhn-TuckerLP Linear programmingMV-OPT Mixed variable optimization

problemNLP Nonlinear programmingPSO Particle swarm optimizationQP Quadratic programmingRBDO Reliability-based design

optimizationSA Simulated annealingSLP Sequential linear programmingSQP Sequential quadratic

programmingTS Traveling salesman

(salesperson)

xvi