neal ford emergent design and evolutionary architecture

ThoughtWorksThoughtWorks

NEAL FORD software architect / meme wrangler

ThoughtWorks

nford@thoughtworks.com 3003 Summit Boulevard, Atlanta, GA 30319

www.nealford.com

www.thoughtworks.com

memeagora.blogspot.com

emergent design & evolutionary architecture

http://www.ibm.com/developerworks/java/library/j-eaed1/index.html?S_TACT=105AGX02&S_CMP=EDU

http://tr.im/nf_ead_all

agenda

things to think about

emergent evolutionary

Emergent, a. [L. emergens, p. pr. of emergere.]

1. Rising or emerging out of a fluid or anything that covers or conceals; issuing; coming to light. [1913 Webster]

2. Suddenly appearing; arising unexpectedly; calling for prompt action; urgent. [1913 Webster]

spectrum of design

"Pure" Waterfall

Emergent Design

Agile

BDUFCowboyHacking

Some DUF

big design up front

Project Planning/Estimation

Requirements

Use Cases/Functional Specs

Design Specifications

Code

Test

Fix/Integrate $

who thought this was a good idea?

emergent (hardware) design

finding abstractions & patterns

domain patterns

nomenclature

technical patterns

patterns describe effective abstractions

Patterns

patternsidiomatic

business processes change radically and often

speculation without facts

YAGNI!

spike solutions

how do you know when?

abstracting too early

experience helps

emergent design

what is software design?

Jack C. Reevesfall 1992, c++ journal

http://www.developerdotstar.com/mag/articles/reeves_design.html

“what is software design?”

what is the design document in software?

“The final goal of any engineering activity is some type of documentation”

“When the design effort is complete, the design documentation is turned over to the manufacturing team.”

the source code

software “engineering”

“...software design is easy to create, at least in the mechanical sense.”

“...software is cheap to build. It does not qualify as inexpensive; it is so cheap it is almost free”.

manufacturing == build process

“Given that software designs are relatively easy to turn out, and essentially free to build, an unsurprising revelation is that software designs tend to be incredibly large and complex.”

source == design

“Coding is design, testing and debugging are part of design, and what we typically call software design is still part of design.”

“...it is cheaper and simpler to just build the design and test it than to do anything else.”

“The overwhelming problem with software development is that everything is part of the design process.”

“Software may be cheap to build, but it is incredibly expensive to design.”

source == design

technical abstractions

discovering design in code

finding effective abstractions

ability to harvest idiomatic patterns

problem domain abstractions

emergent design

things that obscure

emergent design

complexity

essential vs. accidental complexity

accidental complexityall the externally imposed ways that software becomes complex

essential complexityinherent complexity

Hunting Season

Field Level Security

EJB / Biztalk

Essential Accidental

examples

technical ! ! ! debt

technical debt

Code Base Complexity principal

interest

effort for new features

technical debt

demonstration trumps discussion

you must convince someone technical debt exists...

...start a conversation about repayment

negotiating repayment

convincing metrics

rampant genericness

increases software entropy

“if we build lots of layers for extension, we can easily build more onto it later”

generic obfuscation

accidental complexity

genericness

emergent design enablers

atomic understanding of intent

more about design than testing

design will emerge from tests

less accidental complexity

better abstractions

test driven design

perfect number case study:

∑ of the factors == number(not including the number)

∑ of the factors - # == #

tdd forces you to think about every little thing

test after doesn’t expose design flaws as early

encourages refactoring what’s not right

tdd vs test-after

regularly fix obsolescent abstractions

collective code ownership

fix broken windows whenever you see them

code should get stronger with age

prudently refactor aggressively

refactoring

refactoring to harvest idiomatic patterns

cyclomatic complexitymeasures complexity of a function

V(G)= e - n + 2V(G) = cyclomatic complexity of Ge= # edgesn= # of nodes

start

if (c1)

f1() f2()

if (c2)

f3() f4()

end

start

if (c1)

f1() f2()

if (c2)

f3() f4()

end

7

65

4

32

1

8

7

65

4

3

2

1

nodes

edges

determines what is the “hard, crunchy center” of your code base

∑ of how many classes use this class

incoming references

measure with CKJM or other metrics tools

afferent coupling

struts 2.x

UIBean evaluateParams()

evaluate.*Params ?

./org/apache/struts2/components/AbstractRemoteCallUIBean.java

./org/apache/struts2/components/Anchor.java

./org/apache/struts2/components/Autocompleter.java

./org/apache/struts2/components/Checkbox.java

./org/apache/struts2/components/ComboBox.java

./org/apache/struts2/components/DateTimePicker.java

./org/apache/struts2/components/Div.java

./org/apache/struts2/components/DoubleListUIBean.java

./org/apache/struts2/components/DoubleSelect.java

./org/apache/struts2/components/File.java

./org/apache/struts2/components/Form.java

./org/apache/struts2/components/FormButton.java

./org/apache/struts2/components/Head.java

./org/apache/struts2/components/InputTransferSelect.java

./org/apache/struts2/components/Label.java

./org/apache/struts2/components/ListUIBean.java

./org/apache/struts2/components/OptionTransferSelect.java

./org/apache/struts2/components/Password.java

./org/apache/struts2/components/Reset.java

./org/apache/struts2/components/Select.java

./org/apache/struts2/components/Submit.java

./org/apache/struts2/components/TabbedPanel.java

./org/apache/struts2/components/table/WebTable.java

./org/apache/struts2/components/TextArea.java

./org/apache/struts2/components/TextField.java

./org/apache/struts2/components/Token.java

./org/apache/struts2/components/Tree.java

./org/apache/struts2/components/UIBean.java

./org/apache/struts2/components/UpDownSelect.java

find . -name "*.java" | xargs grep -l "void evaluate.*Params" > pbcopy

!

fixing parameters

22 * 3 = 66

66 - 9 = 57lines of

obscuring duplication

most comments don’t help

if code is design, readable design matters

complex languages hurt readability

always (potentially) out of date

not executable

expressiveness

idiomatic “unit of work” pattern

without closures (java)

with closures (groovy)

with closures (ruby)

object-oriented

imperative

structured / modular

anti-objects

functional

abstraction styles

“The metaphor of objects can go too far by making us try to create objects that are too much inspired by the real world. “

“...an antiobject is a kind of object that appears to essentially do the opposite of what we generally think the object should be doing.”

collaborative diffusion

architecture

Evolution, n. [L. evolutio an unrolling: cf. F. ['e]volution evolution

1: a process in which something passes by degrees to a different stage (especially a more advanced or mature stage)

describes the coarse-grained pieces that compose an application

application architecture

JSR 277, the java module system...abandonware

when was the last time you downloaded a single class?

the unit of reuse in java is the library

implemented by ivy & maven

JSR 294 (superpackage)...IN JAVA 7!

framework level architecture

enterprise architectureconcerns itself with how the enterprise as a whole

(which usually means the applications running inside a large organization) consumes applications

enterprise architecture == city planning

application architecture == building plan

extant definitions

“The architecture of a software system (at a given point in time) is its organization or

structure of significant components interacting through interfaces, those components being

composed of successively smaller components and interfaces.'"

post on the XP mail list

technical definition

Ralph Johnson, rebutting the original post

"In most successful software projects, the expert developers working on that project have a

shared understanding of the system design. This shared understanding is called "architecture."

This understanding includes how the system is divided into components and how the

components interact through interfaces.'"

social definition

Architecture is about the important stuff.

Whatever that is.

Martin Fowler’s definition

Stuff that's hard to change later.

Martin Fowler, in conversation

There should be as little of that stuff as possible.

architecture

design

architectural considerations

politics of architecture

build or buy

“...we just need to tweak it with a few customizations”

radically unique across similar businesses

“you can buy this generic business process software...”

software can provide strategic business advantage

myth

business processes are not commoditizable

the next 10% is possible but difficult

for framework X:

80% of what the user wants is fast & easy

users want 100% of what they want

the last 10% is impossible

Dietzler’s Law

...but vendors hate it

flash-back to java web development before j2ee

standards helped developers tremendously...

contrast j2ee & sql

the price of commodity software quickly approaches $0

standards-based vs. standardized

even the open source ESBs suffer from this

big enterprise package software touts standards-based

held together with highly proprietary glue

consider the impact on your overall architecture

not likely to change

ESB: standards-based but not standardized

postlude

other artifacts aid in creating code

all artifacts besides code are transient

irrational artifact attachment

design == code

How can I make my decision reversible?

Do I need to make this decision now? What can I do to

allow me to defer the decision?

last responsible moment

ThoughtWorks

This work is licensed under the Creative Commons Attribution-Share Alike 3.0 License.

http://creativecommons.org/licenses/by-sa/3.0/us/

for information & more cool stuff, visit thoughtworks.com

NEAL FORD software architect / meme wrangler

ThoughtWorks

nford@thoughtworks.com 3003 Summit Boulevard, Atlanta, GA 30319

www.nealford.com

www.thoughtworks.com

memeagora.blogspot.com

resources

ThoughtWorks

http://tinyurl.com/nf-ead

Emergent Design & Evolutionary Architecture series:

appendix aTest-after vs. TDD Perfect Number Finder

appendix bMartin Fowler’s Who Needs an Architect