framework engineering

Son YoungSuindigoguru@gmail.com

Microsoft MVPEvaCast Leader

Devpia Architecture SysopLoad to Architect (http://www.arload.net)

FrameworksFrameworks define define “semi“semi--complete” applicationcomplete” applicationthat embody domainthat embody domain--specific object structures and functionality.specific object structures and functionality.

Class Library Class Library Component Component ArchitectureArchitecture

App SpecificApp SpecificLogicLogic

OO DesignOO Design

EventEventLoopLoop

DATABASEDATABASE ADTsADTs

MATHMATH NETWORKINGNETWORKING

GRAPHICSGRAPHICS GUIGUI

SingletonSingleton StrategyStrategy

ReactorReactor AdapterAdapter

StateState Active ObjectActive Object

InvocationsInvocations

SelectionsSelections

Application BlockApplication Block

Design PatternDesign PatternDATABASEDATABASE

SingletonSingleton

GRAPHICSGRAPHICS

AdapterAdapter

EventEventLoopLoop

ReactorReactor

NETWORKINGNETWORKING

Active ObjectActive Object

GUIGUI

StateState

App SpecificApp SpecificLogicLogic

MATHMATH

ADTsADTs

CallbacksCallbacks

InvocationsInvocations

Application Application FrameworkFrameworkComponent ArchitectureComponent Architecture

Avoid Duplication Productivity

Do you Do you wannawanna make make good framework?good framework?

Motivation

Organization

Architecture

Planning

Design

Development

Resources

“Framework Engineering”, TechED 2007 Europe

“Framework Design Guidelines” , Addison Wesley

Krzysztof Cwalina

Program Manager on .NET Framework Team

Organization

Planning

ArchitectureDesign

Development

Your APIs

D

O

P

A

V

The most powerful design toolThe most powerful design tool

O

Scope

Cost Time

Scope

Cost

Time

Organization

DODO understand how understand how organizational structure, organizational structure, culture, and decision making processesculture, and decision making processes impaimpact your product. ct your product.

O

If you have 4 groups working on a compiler,

you’ll get a 4-pass compiler.

If you have four groups working on a compiler,

you’ll get a 4-pass compiler.

Organization Structure

Software Structure

1• Define the business mission• Define the business mission

2• Learn the business process from business owners• Learn the business process from business owners

3• Re-engineer these process to fit the mission• Re-engineer these process to fit the mission

4

• Structure the IT organization to support the reengineered business processes.

• Structure the IT organization to support the reengineered business processes.

Voluntary

Familial

Peremptory

Small Team

Simple Design

Consistency Design

Focus on 80/20 Rules

Large Team

Powerful Design

Lack Consistency

Remove Requirements

Customer-Focused

End-2-End Scenarios

Technology-Focused

Long Lasting Architecture

Individuals Empowered -> Time to Market

Hierarchy -> Interoperability and Integration

Consensus -> Team Building

Ensuring we are building the right thing Ensuring we are building the right thing

P

Focus: featuresFocus: features

Results: Results: stability, stability,

incremental incremental

improvements, not improvements, not

great great endend--toto--end end

scenariosscenarios

Peanut ButterPeanut Butter

Focus: scenarios Focus: scenarios

Results: Results: Excitement, Excitement,

breakthroughs, but breakthroughs, but

beware of leaving beware of leaving

existing customers existing customers

behind behind

SkyscrapersSkyscrapers

P

Planning M1 M2

ReleaseTesting

Feature completeVision statement RTM

Technology Preview Beta 1 Beta 2 RC1

Ensuring the long term health of the frameworkEnsuring the long term health of the framework

A

DO manage your dependencies.

A

A Component is a set of types that ship and evolve as a unit. Componentization is a process of organizing types into components, with explicitly designed and controlled dependencies between componexplicitly designed and controlled dependencies between componentsents. NOTE: Componentization is different from assembly factoring (i.e. an assembly might have more than one component)

A

API Dependency: Component A has an API dependency on component B, if a type in B shows in the publicly accessible (public or protected) API surface of a type in A. This includes:

Base types and implemented interfacesGeneric parameter constraintsReturn types and parameters of membersApplied attributesNested types

Implementation Dependency: If a type in A uses a type in B in its implementation.

Hard Dependencies (required to run)Soft Dependencies (optional)

Circular Dependency occurs when component A depends on component B and component B depends on component A (even indirectly).

Note: Two (or more) components with a circular API dependencies can be considered a single component for all practical purposes.

A

A

Layering is a process of organizing components in layers and enforcing dependency rules between components in these layers.

Types in a component can freely depend on each other Cross-component dependencies must be carefully controlled A component can freely take dependencies on components in a lower layer A component must not have hard dependencies on components in higher layers. A component should avoid soft dependencies on components in higher layers. Dependencies between components of the same layer must be carefully managed. [NOT

E: we have an approval process for these] In general it’s good to minimize dependencies, if it does not create to much duplic

ation (bloat)

A

NDepend - http://www.ndepend.com

Primitives

Abstractions

Reusable Components

Very little policy (behavior design decisions)

Stable design

Commonly appear in publicly accessible APIs

Almost impossible to evolve/change design; any design changes have huge breaking change impact on other APIs

Example: Int32, String

A

Interfaces, abstract classes, some concrete classes with extensive virtual surface.

Similar to Primitives (show in APIs), but usually have more policy (though less than libraries)

The hardest types to design rightWith the exception on a few historically well understood abstractions (lists, streams, etc.), abstractions with more than 2-3 members are rarely successful.

Difficult to evolve

Glue between parts of the frameworkThrough polymorphism

Very useful for decoupling components

Examples: Stream, IEnumerable<T>

A

Perform operations on primitives and abstractions (or the system)

Almost never passed around

Change often as frameworks evolveXmlDocument (Fx 1.0 – XML DOM)

XPathDocument (Fx 2.0 - XPath)

XDocument (Fx 3.5 – Linq to XML)

Relatively easy to evolve (if designed and used properly); they can be simply replaced.

Examples: XmlDocument, EventLog, SerialPort

A

Rich APIs with lots of features, thus with lots of dependenciesGreat usability, poor evolvabilityE.g. Object.GetType() – Every object has very easy access to its type, but also every object depends on ReflectionGood for higher level components, not for the core of a platformNOTE: “Component” is an overloaded term. In this context it does not have anything to do with “componentization.” Unfortunately, COD is already an established term.

A

A.K.A. “Handle based design” (functional)

Great evolvability, poor usability (sometimes)

Low level sable primitives + high level reusable components with limited dependencies other than to the primitives

E.g. Type.GetType(object) – works, but not as convenient as Object.GetType

A

Members with “heavy” dependencies can be extension methods for primitives in a separate component.

This is essentially functional programming// low level assembly with no dependency on globalizationnamespace System {

public struct Decimal {public string ToString(); // culture independent

}}

// higher level assemblynamespace System {

public static class DecimalFormatter {// decimal point character is culture sensitive public static string ToString(this Decimal d, string format);

} }

Note: Same namespace makes the API easy to use A

DO balance advances with compatibility.

A

Cross-Version Compatibility: code written for a version of a redist works on a different version of the same redist.Cross-Redist Compatibility: code written for a redist works on a different redist.Backward Compatibility: code written for a version of a redist works on a newer version of the same redist.Forward Compatibility: code written for a version of a redist works on a previous version of the same redist.

A

Binary Compatibility: a binary runs on a different version or a different redist than what it was build for without recompilation.

Source Compatibility: source code compiling on a version of a redist can be recompiled without changes on a different version or a different redist.

API Compatibility: Stronger than source. Weaker than binary. Source compatibility allows for some changes in APIs (e.g. covariant changes to input parameters). API Compatibility does not.

A

Define what’s a “breaking change”

This definition depends on the objectiveE.g. Enable portable code between Silverlight and .NET Framework

E.g. Enable cross-version portability?

For example, Silverlight interfaces cannot have less members than .NET interfaces, but concrete types can.

A

AVOID duplication and overlap.

A

AVOID duplication and overlap.

A

Problem Space Show and Tell

PLoP – Capable, Productive and Satisfied Patterns for Productivity

http://hillside.net/plop/plop98/final_submissions/P54.pdf

When the new technology is “10x better”

Make sure you understand the impact on the ecosystem

What would happen if the BCL team added a new String?

What’s the migration path for code using the old API?

A

This is where quality happensThis is where quality happens

D

DO design APIs by first writing code samples for the main scenarios and then defining the object model to support the code samples.

D

D

static void Main(string[] args)

{

StreamReader sr = File.OpenText("MyFile.txt");

string s = sr.ReadLine();

while (s != null)

{

s = sr.ReadLine();

Console.WriteLine(s);

}

}

static void Main(string[] args)

{

foreach (string s in File.ReadAllLines("MyFiles.text"))

{

Console.WriteLine(s);

}

}

D

Project -> Add -> Assembly

Download here - http://code.msdn.microsoft.com/fds

Red is removed,

Green is added,

Grey means inherited.

Tools -> Export to Document

DO treat simplicity as a feature.

D

Remove Requirements

Reuse Existing Concepts or APIs

Adjust Abstraction Level

Evolving Framework (Three Example)

D

DO measure, measure, and measure!

D

Performance GoalsBaseline: What do is the best my API could do?

Measure delta from the baseline

Threat ModelsThreat: What is the worst that my component could do?

Mitigate the threats

Same for many other qualities you want your framework to have

D

The bits customers get, … or notThe bits customers get, … or not

V

Main

PU-staging

branch

Feature

branch

Feature

branch

PU-staging

branch

PU-staging

branch

V

AVOID integrating unfinished features.

V

Functional SpecificationDeveloper Design SpecificationTest PlanThreat ModelAPI reviewArchitectural ReviewDependency ManagementStatic AnalysisCode CoverageTesting (Unit and Integration Tests)0 BugsPerformance

V

DO pay your debt.

V

Planning M1 M2

ReleaseTesting

Feature completeVision statement RTM

Technology Preview Beta 1 Beta 2 RC1

Milestone Quality

Initiatives that are hard to do in regular milestones

Large productivity and efficiency improvements

Infrastructure changes

For example, a new source control system

Refactoring of fragile subsystems

Internal implementation documentation

Bugs backlog

V

DO understand how organizational structure, culture, and decision making processes impact your product.

AVOID peanut-butter in Scenario Based Application.

DO manage your dependencies.

DO balance advances with compatibility.

AVOID duplication and overlap.

DO design APIs by first writing code samples for the main scenarios and then defining the object model to support the code samples.

DO treat simplicity as a feature.

DO measure, measure, and measure!

AVOID integrating unfinished features.

DO pay your debt.

Framework Design Guidelines:Conventions, Idioms, and Patterns for Reusable .NET LibrariesKrzysztof Cwalina, Brad Abrams

http://www.gotdotnet.com/team/fxcop

Douglas C. Schmidt (PLoP Editor, POSA 2, 4 Writter)

JAWS: An Application Framework for High Performance Web Systemhttp://citeseer.ist.psu.edu/81775.html (En)http://www.devpia.com/net2/EvaCast/Lecture/?cu=view&r=11 (Kr)

Ralph Johnson (GoF , Design Patterns)

Evolving Frameworkshttp://st-www.cs.uiuc.edu/users/droberts/evolve.html (En)http://arload.wordpress.com/2008/09/15/evolvingframeworks/ (Kr)