practical & philosophical reflections on a life in genetic algorithms

Practical & Philosophical Reflections on a Life in Genetic Algorithms

David E. GoldbergIllinois Genetic Algorithms LaboratoryUniversity of Illinois at Urbana-ChampaignUrbana, IL 61801 USAEmail: [email protected]; Web: http://www.illigal.uiuc.edu

1 © David E. Goldberg 2009

Reflections on a Life in GAs

• 29th year in GAs; 26th year since dissertation. 20th year since GASOML (aside).

• Have been blessed to be part of growth of this field.

• Could easily have been otherwise.• Almost every central turning point

was unlikely event.• Want to reflect on those times

personally, practically & philosophically.

• Guidance for the young (amusement for old?).


Roadmap

What’s a nice civil engineer doing in a place like this?

A cocktail party in Canterbury.One September in Ann Arbor.A professor named Holland.The education of a genetic algorithmist.My philosophical turn & starting a

company.Reflections on existentialism,

paradigms, and the education of engineering and comptuer scientists.

Finding and life’s impedance match.


Once Upon a Time…

• Once upon a time…– There was a civil engineer– working for Stoner Associates – doing hydraulics software for pipelines.

• Was starting to do real-time control &– wondered how human operators– controlled gas pipelines – like you or I drive a car.

• Went to British Hydromechanics Research Association to represent company.


A Cocktail Party in Canterbury

• At the opening reception.• My advisor walks in…• Like the parting of the Red

Sea.• Another prof asks WHEN

will I return for PhD.• Not “cost effective.”• A phone call & a big night.

E. Benjamin Wylie (b. 1928)


One Fine September Day in A2

• First day of classes and was signed up for standard AI course.

• Expert systems were the rage, Prolog was hip, LISP was cool.

• Class was cancelled with little sign on the door.• Hopes and dreams down the drain.• Searched and searched for a replacement.• Found CCS 524, Intro to

Adaptive Systems, taught byJohn Holland.


A Professor Named Holland

• Youngish looking prof:– Talking about biology &

genetics.– Samuel’s checker player.– Schemas and building blocks.– Classifier systems.

• What’s nice civil engineer doing in class like this?

• When was Prof Holland going to get to real AI I could use for pipelines?

• Or maybe this was the real AI.


Education of a Genetic Algorithmist

• 1984 took position in Engineering Mechanics at Alabama, Tuscaloosa.

• Education began then, but there was a lot I needed to learn:

• Focus on 4 core lessons:– Learning to ask– Learning to label– Learning to decompose– Learning to model


Lesson 1: Learning to Ask

• In 1984 had many questions about how GAs work, when they fail?

• Wasn’t experienced in asking good framing questions.

• Key problem: Using GAs to solve engineering problems, but GAs weren’t engineered well.

• Philosophical terms: Socrates 101.

© David E. Goldberg 20099

Socrates (470-399 BCE)

What’s a Good Question?

• Socrates asked variety of questions.• What is truth? What is courage?• More often the critic. Rarely gave answers.• In creative enterprises, many good questions are framing questions:

– Get at heart of the issue.– Help define the problem or elicit definition.– Sometimes cause problem to be represented in novel way or from

unusual or creative perspective.• Fundamental importance of dialectic. Creative process of asking and

answering questions.• GA example from 1985: Alleles, Loci & Traveling Salesman Problem. How

is inversion for orderings, similar to and different from mutation & crossover for alleles?


Lesson 2: Learning to Label

• In the early days, language was nonexistent or unsettled.

• Challenge of being category creator vs. category enhancer.

• Tabula rasa or a green field.• Some borrowed from biology, “fitness,”

“linkage” & “landscape.”• Others invented: “deception,” “niching,”

“abeyance,” • Philosophical terms: Aristotle 101. • Underappreciated as means to

understanding and solving problems. • GA example: Use of term “linkage learning”

leads to practical schemes such as mGA, fmGA, LLGAs, and adaptive EDAs.


Aristotle (384-322 BCE)

Terms Really Do Matter

• Terms gather thoughts under consistent rubrics.

• Can be part of larger taxonomy.

• Defines attention areas.• Can have influence on how

others think.• Catchy or sticky terms

propagate virally.


Lesson 3: Learning to Decompose

• Wasn’t experienced at decomposing big problem into little problems.

• Looked for magic bullets in equations of motion or transform methods.

• 1990 talk by Gary Bradshaw on the Wright Brothers and their explicit decomposition of powered flight.

• Philosophical terms: Descartes 101?


René Descartes (1596-1650)

Design Decomposition for GA Design

• ICGA 1991: Shared “theory” tutorial with Gunar Liepins.

• Need design theory that works:– Understand building blocks (BBs), notions or

subideas.– Ensure BB supply.– Ensure BB growth.– Control BB speed.– Ensure good BB decisions.– Ensure good BB mixing (exchange).– Know BB challengers.

• Will discuss more technically in tutorial tomorrow.


Lesson 4: Learning to Model

Knew quite a bit about modeling mathematically.

Engineers as Pavlovian dogs when it comes to equations.

Didn’t know how to model conceptually:◦ Causal chain.◦ Categorize according to list of types or kinds.

Need to understand problem qualitatively in words and diagrams prior to quantitative modeling undertaken.

Philosophical terms: Hume 101 or Aristotle 102.


David Hume (1711-1776)

A Model of Models

Error, ε

Cost of Modeling, C

Engineer/Inventor

Scientist/Mathematician


What is a “Model?”

Low Cost/High Error

High Cost/Low Error

Unarticulated Wisdom

Articulated QualitativeModel

DimensionalModels

FacetwiseModels

Equations of Motion

The Modeling Spectrum


Marginal Analysis• When should engineer/inventor adopt more

expensive model?• At the margins, when ΔB ≥ ΔC.• Marginal benefit of model to technology under

development must equal or exceed its marginal cost.• To engineer/inventor, artifact is the object of study

models almost always instrumental.• To scientist/mathematician building a model

– may be the object – or instrumental to some other goal (then engineer’s

calculus applies).


Demarcation of Engineering Knowledge

• Realized that I was on philosophical grounds, a demarcation argument.

• Realized that many practices in engineering and CS have proceeded without critical reflection.

• Engineering and CS studied without definition.

• Starts with misleading math-science death march.

• Gives impression that engineering = analysis or “the basics” (math, science, engineering science).

• Ontology, epistemology, reasoning ignored.

• “Design” as abused term & mysterious process.


My Philosophical Turn

• Have turned to philosophy for personal & professional reasons.– Started Engineering and Technology Studies at Illinois or ETSI

(with Michael Loui).– Co-chaired 2007 and 2008 Workshop on Philosophy &

Engineering (WPE) at TUDelft and Royal Academy of Engineering.– Started engineering reflections track at Society for Philosophy &

Engineering.– Co-Founded Illinois Foundry for Innovation in Engineering

Education (2007, 2008).– Co-chaired Summit on the Engineer of the Future 2.0 at Olin

College.• Turn had its roots in starting a company.


Conceptual Modeling at ShareThis

• Was asked to join team of ShareThis (then Nextumi) in 2004.

• Create consumer chromosome inspired by GASOML.

• Did tech work, but also worried a lot about modeling “creepiness.”

• Models were conceptual.• Ray Price, tech visionary

research & a course.• Design of Innovation, explored

qual-quant divide.


Some Philosophical Reflections

• The existential pleasures of engineering.• Kuhn, paradigms, and all that.• Is GEC stuck in a paradigm or paradigms?• Is education of engineers and computer scientists

stuck in paradigm?• With so many calls for educational change, how

come we’re still stuck?


Existential Pleasures of Engineering

• Slide title taken from book by Samuel Florman.

• Making cool technology is fun.• Existential philosophers: Life is lived.

Dasein, beings in time in the process of being.

• We choose. Things happen. We choose again.

• Thought of careers as planned. Tracing my career path as example.

• Not unlike genetic algorithms: Interesting mix of randomness and choice resulting in the solutions that become. Martin Heidegger (1889-1976)


Kuhn, Paradigms & All That

• My cocktail party started with me stuck in a “cost effectiveness” mindset.

• “Paradigm” traces to The Structure of Scientific Revolutions in 1962.

• Kuhn argued that science proceeds in fits and starts, not gradually.

• Old paradigms, ways of thinking about the world, are overturned by revolutions, not gradually.

• What ways are we all stuck in paradigms?• What ways is GEC community stuck in its

paradigm?Thomas S. Kuhn (1922-

1996)


Is GEC Locked in a Paradigm?

• What habits of thought productive early in GEC are counterproductive now?– Continued adherence to old religions (GA, ES, EP, GP).– Loose metaphorical operator design without any analysis? A vs.

B comparisons with little basis.– Rigorous theory & no consideration of design implications?– Lack of progress in examining or contributing to understanding

biological mechanism. • Oftentimes progress comes from new influences: What field or

disciplines are we not collaborating with that would help make progress?

• Interested in neuroscience, philosophy, GAs & consciousness.


Are Engineering & CS Ed Stuck in Paradigm?• Paradigm of tech academy is from the cold war.• Following assumptions sacrosanct:

– Basic engineering science key to success.– Government funds superior to industry $$$.– Demonstrate mettle as individuals with peer-reviewed journal

papers in specialty.

• Question any stare, derision & ridicule.• These beliefs are not scientific. • Paradigm of 50s-present.• Code words: “the basics,” “rigorous,” & “soft.”• Invoking code words not an argument.26 © David E. Goldberg 2009

The Missing Basics

• Have taught 19 years in industrially sponsored senior design course.

• After 4 years students don’t know how to– Question: Socrates 101.– Label: Aristotle 101.– Model conceptually: Hume 101 & Aristotle 102.– Decompose: Descartes 101.– Measure: Bacon-Locke 101.– Visualize/draw: da Vinci-Monge 101.– Communicate: Newman 101

• List starts as before in education of Gamist.• Call these the missing basics (MBs) vs. “the basics” =

math, sci, & eng sci.• Missing basics are in some sense more basic than “the

basics.”• Why does engineering education backfill these skills in

practice?Socrates (470-399 BC)


The Missing Basics as Rosetta Stone

• Missing basics key to – Engineering and CS ed reform– Liberal ed reform– Interdisciplinary research– Lifelong learning

• If math & science the center how do humanists and scientists talk?

• Wrong turn at the Enlightenment.• Toulmin’s argument that geometry is

not a good general epistemological model.


An Academic NIMBY Problem

NIMBY = Not in my backyard.

“Reform is fine…”“….as long as you don’t

change my course.”Politics of logrolling: You

support my not changing. I support your not changing.

Even when agreement for change is acknowledged, almost all specific changes are resisted.


iFoundry: A Pilot Incubator for Change

• Less planning more dot connecting.• iFoundry = Illinois Foundry for Innovation in Engineering Education:

– Separate pilot unit/incubator. Permit change.– Collaboration. Large, key ugrad programs work together. Easier approval if

shared. – Connections. Hook to depts, NAE, ABET (?), industry. – Volunteers. Enthusiasm for change among participants. – Existing authority. Use signatory authority for modification of curricula for

immediate pilot. – Respect faculty governance. Get pilot permission from the dept. and go

back to faculty for vote after pilot change– Assessment. Built-in assessment to overcome objections back home. – Scalability. Past attempts at change like Olin fail to scale at UIUC and other

big schools.


A Life in Genetic Algorithms

Events• Bumped into GAs by accident.• Joined field at time of growth.• Fluids training as disciplinary

grounding in complexity.• Wrote a book I was told not to

write.• Became philosophical in a

action-oriented field.• Took on reform effort not

admired by peers.

Lessons?• Important things can be

random.• Opportunity is knocking? Will

you answer the door?• Being appropriately different

can be beneficial.• Authority figures are not

necessarily right or wise.• Exploring the unexplored can

yield interesting insights.• Sometimes important jobs are

not valued by others.31 © David E. Goldberg 2009

Finding a Life’s Impedance Match

• What a contrarian!!• Not suggesting that others should

follow.• Aristotle talked about virtues

leading to happiness or eudaimonia.• About fulfilling your potential.• New positive psychology takes up

these ancient themes.• Been blessed to be able to do things

I found to be interesting and important.

• Hope you are blessed, too.


More Information• Goldberg, D. E. (2002). The design of

innovation: Lessons from and for competent genetic algorithms. Boston, MA: Kluwer Academic Publishers.

• Lab: www.illigal.org

• iFoundry: www.ifoundry.illinois.edu

• Philosophical writing: http

://philsci-archive.pitt.edu/ (search for

“Goldberg”).

• Powerpoint: www.slideshare.net/deg511

• YouTube: www.youtube.com/illinoisfoundry


http://www.illigal.org/

http://www.ifoundry.illinois.edu/

http://philsci-archive.pitt.edu/



http://www.slideshare.net/deg511

http://www.youtube.com/illinoisfoundry