ESSENCE-DRIVEN ADAPTIVE

SOFTWARE ENGINEERING

Dr. June Sung Park, Invited Professor, KAIST / President, SEMAT

http://june-sung-park.flavors.me/

ESSENCE CONFERENCE

ESSENCE KERNEL

Alpha

Activity Space

Competency

2

Read [5], [12]

ALPHA

Alpha represents things to

deal with in any software

engineering project.

3

Cu

sto

me

rSo

luti

on

End

eav

or

Opportunity Stakeholders

RequirementsSoftware System

Work Team

Way of Working

<provide

use an

dco

nsu

me>

<pro

du

ces

scop

es and

con

straints>

focu

ses>

<fulfills

<performs and plans

<set up

to ad

dress

Sup

po

rt>

ALPHA STATE AND CHECKLIST 4

ALPHA DECOMPOSITION AND EXTENSION

An alpha may have lower-level, more granule

sub-alphas whose states contribute to and drive

the state of the super-alpha.

• Association between super-alphas and sub-alphas

can be many-to-many.

An alpha may be Extended (i.e., have the values

of its attributes be changed) in the context of a

Practice (such as Scrum).

5

Alpha Sub-Alpha

Alpha Extended Alpha

ESSENCE KERNEL EXTENSION

Patterns can arrange language

elements into arbitrary

meaningful structures.

Resources can be attached to any

language element.

Tags add user defined

information to any language

element.

User-Defined Types detail,

explain, and constrain the proper

usage of particular patterns,

resources, or tags.

6

ESSENCE LANGUAGE 7

STATE OF SOFTWARE ENGINEERING PROJECT 8

How is the

project

going?

Oh, I guess

it’s going

OK?

STATE OF SOFTWARE ENGINEERING PROJECT 9

Yeah, this

is the

current

state.

Really?

You’re

sure?

ALPHA STATE CHECKLIST

I use state

checklists.

How do

you know

where we

are?

Alpha State Card State ChecklistEssence Alpha Card

GOAL STATE

So what’s

next to do?

Well, I will show

you how I

determine the

next activities

to achieve the

next goal state.

Current State Goal State

CHOOSING ACTIVITIES TO FULFILL THE GOAL STATE

Before we proceed,

you need to

understand how

software engineering

practices are

mapped to the

Essence kernel.

Software Engineering PracticeEssence Kernel

Hmm …

METHOD DESCRIPTION IN ESSENCE LANGUAGE

There are probably hundred

thousands of methods applied in

SE projects worldwide.

There are about 300 well known

practices reusable across projects.

Those practices can be described

using a common notation--

Essence kernel and language.

A project method can be

composed of practices.

13

Practices

Methods

Custom Method A Custom Method Barecomposedof

aredescribedusing

are definedin terms of

Essence Kernel

Essence Language

OMGStandard

ESSENCE KERNEL AND METHOD 14

Role

Alpha

Alpha State

Activity Space

Competency

Work Product

Activity

<has

targ

ets>

pro

du

ces

/u

pd

ates

>

< defines

Practice

< is

co

mp

ose

d o

f

Method

profiles>

is q

ual

ifie

dto

p

erfo

rm>

Read [18]

PRACTICE DESCRIPTION IN ESSENCE LANGUAGE

A software engineering practice can be

described in Essence kernel and

language by mapping:

• work products to Alphas

• activities to Activity Spaces

• roles to Competencies

15

Practice

Work Product

Activity

Role

Essence Kernel

Alpha

Alpha State

Activity Space

Competency

Read [19]

ACTIVITY SPACE

Activity spaces are containers of activities performed in a project.

• An activity may be a part of another activity forming a work breakdown structure.

The association between activity spaces and activities can be many-to-many.

16

Cu

sto

me

rSo

luti

on

End

eav

or

Explore Possibilities

Understand Stakeholder Needs

Ensure Stakeholder Satisfaction Use the System

Understand the Requirements

Shape the System

Implement the System

Test the System

Deploy the System

Operate the System

Prepare to do the Work

Coordinate the Activity Support Team Track Progress Stop the Work

ACTIVITY SPACE AND ALPHA STATE

Pre and post conditions of each activity space are suggested in terms of alpha states in the kernel.

17

Read [19]

ACTIVITY AND STATE TRANSITION

By mapping each activity to one or

more Activity Spaces the activity

inherits “default” state transitions (i.e.,

starting Alpha States and target

Alpha States) and associated

checkpoints (i.e. criteria of done).

By tailoring the checkpoints for the

default target states we can generate

an Activity Checklist.

18

Activity Space

Competency

Activity

Alpha

Alpha State

Work Product

<has

targ

ets> p

rod

uce

s/

up

dat

es>

Read [16]

COMPUTING ACTIVITY CHECKLIST 19

Read [16]

COMPUTING NEXT ACTIVITIES TO REACH THE GOAL STATE 20

Read [16]

AUTOMATION TOOL: ESSENCIA

OK, I see

the value

of Essence

now.

Well, I have shown you how I

determined the next activities to

perform to achieve the next goal

state.

This can be automated if we use a

tool that supports the description

of practices using the Essence

kernel, and build a database of

checklists for alpha states.

PRACTICE DESCRIPTION APPROACH

1. Build an Ontology of the Terms used in the Practice

Parse the text description of the Practice to build a Glossary.

Classify the Terms in the Glossary into Work Products, Activities, Roles, etc.

Add missing Terms such as activities for producing or updating work products and vice versa.

2. Map the Terms to Essence Language Elements.

Determine alphas, alpha states and checkpoints corresponding to each work product.

Determine activity spaces, beginning and target alpha states, target checkpoints corresponding to each activity.

Determine competencies required of different roles.

3. Decompose and Extend Essence Kernel Elements to represent detailed concepts, composite constructs and complex relationships.

Define sub-alphas, sub-activity spaces, patterns, resources and tags to represent concepts in the practice.

22

Build Practice Ontology

Map Terms to Essence Language Elements

Decompose and Extend Essence Kernel Elements

if necessary

SCRUM PRACTICE 23

Development Team

Task Breakdown

Product Increment

Jeff Sutherland and Ken Schwaber, The Scrum Guide, 2013. (http://www.scrumguides.org/)

SCRUM GLOSSARY

Key Terms ClassificationRelationship

Added TermsRole Activity Work Product

Daily Scrum Activity Development Team Sprint Plan, Total Work RemainingDefinition of Done Work Product Sprint Retrospective Increment, Product Backlog RefinementDeveloper RoleDevelopment Team Role Daily Scrum Sprint Backlog, Development Work, Increment

Development Work ActivitySprint Backlog, Development Work Plan, Work Unit, Increment

Development Work Plan

Improvement Plan Work Product Sprint RetrospectiveIncrement Work Product Sprint Review Sprint Plan, Sprint Goal, Sprint Backlog, Definition of Done

Product Backlog Work Product Product OwnerProduct Backlog Refinement, Sprint Review

Product Backlog ItemProduct Backlog Creation

Product Backlog Item Work Product Product Backlog Product Backlog Refinement Activity Product Backlog

Product Owner RoleProduct Backlog Creation, Product Backlog Refinement, Sprint Review

Product BacklogProduct Backlog Creation

Scrum Event Composite ActivityScrum Master Role Sprint RetrospectiveScrum Team Work Product PO, DT, SMSprint MilestoneSprint Backlog Work Product Development Team Product Backlog, Sprint Goal, Development WorkSprint Goal Work Product Sprint Planning Sprint PlanningSprint Plan Composite Work ProductSprint Planning Activity Sprint PlanSprint Retrospective Activity Scrum Master Sprint Plan, Definition of Done,

Sprint Review Activity Stakeholders, Increment, Product Backlog, Total Work Remaining, Sprint Plan

Stakeholders Role Sprint ReviewTotal Work Remaining Work Product Sprint Review, Daily ScrumWork Unit Work Product Sprint Backlog, Development Work

24

SCRUM ONTOLOGY 25

SCRUM TO ESSENCE KERNEL MAPPING 26

Scrum

Product Backlog Product Backlog Item

Sprint Backlog

Scrum Team

Development Work Plan Work Unit

Increment

Total Work Remaining

Improvement Plan

Requirements

Software System

Work

Team

Way of Working

Sprint Goal

Definition of Done

Opportunity

Product Backlog Refinement

Sprint Review

Sprint Planning

Daily Scrum

Sprint Retrospective

Ensure Stakeholder Satisfaction

Understand the Requirements

Coordinate the Activity

Support the Team

Track Progress

Product Backlog Creation

Development WorkImplement the System

Test the System

Shape the System

Understand Stakeholder Needs

Explore Possibilities

COMPOSITE CONSTRUCTS IN SCRUM 27

Sprint Review

Sprint Planning

Daily Scrum

Sprint Retrospective

Sprint Plan

Product Backlog Item

Sprint Backlog

Development Work Plan Work Unit

Sprint Goal

produces

provides input to

may change

SprintIncrement

Scrum Event

Development Work

Produces

Conducts

Product Owner

Development Team

Scrum Master

Manages Product Backlog

Creates

Performs

Ensures enactment of Scrum

Scrum Team

ACTIVITY TO ALPHA STATE MAPPING 28

Activity

Alpha States

Activity Spaces

Opportunity Requirement Software System Team Work Way of Working

Ide

nti

fie

d

Solu

tio

nN

ee

de

dV

alu

eEs

tab

lish

ed

Via

ble

Ad

dre

sse

d

Be

ne

fit

Acc

rue

d

Co

nce

ive

d

Bo

un

de

d

Co

he

ren

t

Acc

ep

tab

le

Ad

dre

sse

d

Fulf

ille

d

Arc

hit

ect

ure

Sele

cte

dD

em

on

st-

rab

le

Usa

ble

Re

ady

Op

era

tio

nal

Re

tire

d

See

de

d

Form

ed

Co

llab

o-

rati

ng

Pe

rfo

rmin

g

Ad

jou

rne

d

Init

iate

d

Pre

par

ed

Star

ted

Un

de

rC

on

tro

l

Co

ncl

ud

ed

Clo

sed

Pri

nci

ple

sEs

tab

lish

ed

Fou

nd

atio

nEs

tab

lish

ed

In U

se

In P

lace

Wo

rkin

gW

ell

Re

tire

d

ProductBacklogCreation

Explore Possibilities

Understand Reqts

ProductBacklog

Refinement

Understand St. Needs

Understand Reqts

SprintPlanning

Understand St. Needs

Understand Reqts

Coordinate Activity

DevelopmentWork

Shape the System

Implement / Test

Daily Scrum Track Progress

SprintReview

Ensure St. Satisfaction

Track Progress

Sprint Retro. Support the Team

ACTIVITY DEFINITION CARD

The project method assembled from Essence-powered practices becomes a small deck of cards in their pockets, which team members can easily pull out to discuss the current project state, the goal state to achieve, and the next activities to perform to reach the goal state.

Teams can also discuss areas of improvement by referring to the cards and their associated checklists.

29

Scrum Practice

Opportunity

Viable

Addressed

A usable system that demonstrably addresses the opportunity is available. The stakeholders agree that the available solution is worth deploying. The stakeholders are satisfied that the solution produced addresses the opportunity.

Work

Under Control

Concluded

All outstanding tasks are administrative housekeeping or related to preparing the next piece of work. Work results have been achieved. The stakeholders have accepted the resulting software system.

Sprint Review

Ensure Stakeholder Satisfaction

Track ProgressProduct Owner Development Team Scrum Master Stakeholder

Product Backlog

Sprint Backlog IncrementSprint Goal

WORK PRODUCT TO ALPHA STATE MAPPING 30

Work Product AlphaAlpha State

Begin In Target

Product BacklogRequirements Bounded Acceptable

Opportunity Solution Needed Viable

Sprint Goal Requirements Bounded Coherent

Sprint Backlog Requirements Coherent Acceptable

Definition of Done Requirements Acceptable Fulfilled

Development Work Plan Work Initiated Prepared

IncrementSoftware System Architecture Selected Ready

Work Prepared Concluded

Total Work Remaining Work Started Under Control

Scrum Team Team Seeded Performing

Improvement Plan Way of Working Foundation Established Working Well

WORK PRODUCT TO ALPHA STATE MAPPING 31

Product Backlog Sprint Goal

Sprint Backlog

Definition of Done

Dev Work Plan

Increment

Increment

Scrum Team

ImprovePlan

TWR

WORK PRODUCT DEFINITION CARD

The Activity Definition Cards and Work Product Definition Cardsprovide concise reminders and cues for team members as they go about their daily tasks.

By providing practical checklists, as opposed to conceptual discussions, the Essence-powered practice becomes something the team uses on a daily basis.

This is a fundamental difference from traditional approaches, which tend to overemphasize method description as opposed to method use, and tend to be consulted only by people new to the team.

32

Scrum Practice

Product Backlog Item

Sprint Backlog

Development Work Plan

Work Unit

Requirements

Coherent

Acceptable

The stakeholders accept that the requirements describe an acceptable solution. The rate of change to the agreed requirements is relatively low and under control. The value provided by implementing the requirements is clear. The parts of the opportunity satisfied by the requirements are clear. The requirements are testable.

Work

Initiated

Prepared

Commitment is made. Cost and effort of the work are estimated. Resource availability is understood. Governance policies and procedures are clear. Risk exposure is understood. Acceptance criteria are defined and agreed with client. The work is broken down sufficiently for productive work to start. Tasks have been identified and prioritized by the team and stakeholders. A credible plan is in place. Funding to start the work is in place. The team or at least some of the team members are ready to start the work. Integration and delivery points are defined.

Sprint PlanningUnderstand the Requirements

Coordinate the Activity

Understand Stakeholder Needs

SCRUM WORKFLOW 33

METHOD COMPOSITION 34

Scrum

Product Backlog Refinement

Sprint Review

Sprint Planning

Daily Scrum

Sprint Retrospective

Ensure Stakeholder Satisfaction

Understand the Requirements

Coordinate the Activity

Support the Team

Track Progress

Product Backlog

Product Backlog Item

Sprint Backlog

Scrum Team

Development Work Plan Work Unit

Increment

Total Work Remaining

Improvement Plan

Requirements

Software System

Work

Team

Way of Working

Product Backlog Creation

Sprint Goal

Development WorkImplement the System

Test the System

Shape the SystemDefinition of

Done

OpportunityUnderstand

Stakeholder Needs

Explore PossibilitiesStakeholder

Business Requirements

Agile Modeling

Opportunity

Software Requirement Model

Software Architecture

Business Analysis

Model Storming

Spike

Read [18]

METHOD COMPOSITION 35

Kernel elements additionally covered by Agile Modeling

Kernel elements covered by Scrum

Add XP

Add DevOps

Add SPM

CASE STUDY: RED HAT

Red Hat Architect Team have been using Essence to help standardize its approach to engagements.

They are using alpha state cards in project planning, which provide consistent language and measurable objectives

with which to access the current state, or articulate next steps and goals.

They created a dashboard website to record and report the current alpha states of engagement projects, which

facilitated high-level project governance.

Essence was also used to build the Red Hat Architectural Framework which provides a framework to collate and

index established approaches: techniques, activities, practices and methods. This enables consultants to find

suitable approaches when seeking to achieve goals expressed in terms of Essence.

Red Hat also applies Essence to analysing existing methods and developing more rounded practices. This type

of analysis has proven useful in finding the holes in existing processes and reworking a process to support a

progressive state-based model that is easier to track and apply.

36

Read [23]

CASE STUDY: FUJITSU UK

Fujitsu UK has been using Essence, in particular alpha state checklists, in iteration planning with customers

They also used Essence to analyse, compare, standardize and integrate various Design and Build Methodologies

(such as Architecture DBM, Applications DBM, Infrastructure DBM, Service DBM, etc.)

With all the disruptive changes occurring in IT today including cloud, mobile, big data and IoT, Fujitsu felt they

needed a common framework to standardize their methodologies and applied the Essence kernel successfully to

that end.

The methodologies were further improved so that quality assurance involved checking alpha states rather than

activity completions.

Fujitsu has 165 thousand employees across most geographies in the world. Fujitsu is now working on

developing a common set of global standard methodologies and processes using the Essence as the common

framework.

37

Read [11]

CONCLUSION

You can use Essence kernel to:

Describe practices

Merge them into a project method

Build an enterprise method architecture

Monitor health and progress of the project

Set up an enterprise-wide dashboard for monitoring all

ongoing projects

Adaptively determine project goals and activities based

on the current state assessment.

38We’d better

learn and

use Essence.

I think so, too. It

really makes

defining and using

methods easy.

Read [7], [10], [18], [20]

REFERENCES

1. D. Cunningham, “Enabling Fujitsu’s Industrialized Delivery of Application Services,” OMG Essence Information Day, Berlin, 2013. (http://www.omg.org/news/meetings/tc/berlin-13/special-events/essence-pdfs/S7-Cunningham.pdf)

2. B. Elvesæter, G. Benguria and S. Ilieva, “A Comparison of the Essence 1.0 and SPEM 2.0 Specifications for Software Engineering Methods,” Workshop on Process-Based Approaches to Model-Driven Engineering, Montpellier, France, 2013. (http://dl.acm.org/citation.cfm?id=2489835)

3. Ivar Jacobson International, “Asian Telecommunications Equipment Vendor Successfully Achieves Rapid and Sustainable Agile Transformation,” 2014. (http://www.ivarjacobson.com/uploadedFiles/Pages/Knowledge_Centre/Resources/Case_Studies/Resources/AsianTelecomm1.pdf)

4. I. Jacobson, P.-W. Ng, P. E. McMahon, I. Spence and S. Lidman, “The Essence of Software Engineering: The SEMAT Kernel,” Communications of the ACM, Vol. 55, pp. 42-29, Dec. 2012.

5. I. Jacobson, P.-W. Ng, P. E. McMahon, I. Spence and S. Lidman, The Essence of Software Engineering: Applying the SEMAT Kernel, Addison-Wesley, 2013.

6. I. Jacobson, I. Spence and P.-W. Ng, “Agile and SEMAT-Perfect Partners,” Communications of the ACM, Vol. 56, pp. 53-59, Nov. 2013.

7. I. Jacobson, P.-W. Ng, I. Spence and P. E. McMahon, “Major-League SEMAT—Why Should an Executive Care?” Communications of the ACM, Vol. 57, pp. 44-50, April 2014.

8. I. Jacobson and E. Seidewitz, “A New Software Engineering,” Communications of the ACM, Vol. 57, pp. 36-41, Dec. 2014.

39

REFERENCES

9. A. McDonough, “Munich Re and Essence – Kernel and Language for Software Engineering Methods: A Case Study,” OMG, 2014. (http://www.omg.org/news/whitepapers/Munich_Re_Essence_Case_Study-2014-12-01_JP.pdf)

10. P. E. McMahon, 15 Fundamentals for Higher Performance in Software Development, Leanpub, 2014.

11. S. Nadin, “Using Essence to Deliver Together: Practical Experience at Fujitsu,” ” Essence-in-Practice Conference in OMG Technical Meeting, Berlin, 2015. (http://www.omg.org/news/meetings/tc/berlin-15/special-events/essence-presentations/nadin.pdf)

12. Object Management Group, Essence—Kernel and Language for Software Engineering Methods 1.0, November 2014. (http://www.omg.org/spec/Essence/)

13. J. S. Park, “Essence Kernel-Based Enterprise Method Architecture,” OMG Essence Information Day, Berlin, 2013. (http://www.omg.org/news/meetings/tc/berlin-13/special-events/essence-pdfs/S5-Park.pdf)

14. J. S. Park, I. Jacobson, B. Myburgh, P. Johnson and P. E. McMahon, “SEMAT Yesterday, Today and Tomorrow,” SEMAT, 2014. (http://semat.org/wp-content/uploads/2014/12/SEMAT-Yesterday-Today-and-Tomorrow-v1.0.pdf).

15. J. S. Park, “Essence-Based Adaptive Software Engineering,” Pre-Conference Tutorial in the Fourth International Conference on Emerging Applications of Information Technology (EAIT), Indian Statistical Institute, Kolkata, India, Dec. 19, 2014. (http://www.scribd.com/doc/251364248/Essence-Tutorial-JP)

16. J. S. Park, “Essence-Based Goal-Driven Adaptive Software Engineering,” General Theory of Software Engineering (GTSE) Workshop in theInternational Conference on Software Engineering, Florence, Italy, May 18, 2015. (http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7169393&filter%3DAND(p_IS_Number%3A7169380)

40

REFERENCES

17. J. S. Park, “Essence-Powered Scrum: A Generic Approach to Describing Practices Using Essence Kernel and Language,” Essence-in-Practice Conference in OMG Technical Meeting, Berlin, Germany, June 18, 2015. (http://www.omg.org/news/meetings/tc/berlin-15/special-events/essence-presentations/park.pdf)

18. J. S. Park, “Software Engineering in the Context of Business Systems—How Essence can Help,” in Ivar Jacobson and Harold Lawson (ed.) Software Engineering in the Systems Context, College Publications: London, 2015. (ISBN 978-1-84890-176-6)

19. J. S. Park, P.E. McMahon and B. Myburgh, “Scrum Powered by Essence,” ACM SIGSOFT Software Engineering Note, Nov. 2015 (forthcoming).

20. C. Péraire and T. Sedano, "State-Based Monitoring and Goal-Driven Project Steering: Field Study of the SEMAT Essence Framework",International Conference on Software Engineering, Hyderabad, India, 2014. (http://repository.cmu.edu/cgi/viewcontent.cgi?article=1147&context=silicon_valley)

21. B. Perkens-Golomb, “Applying SEMAT concepts at Munich Re: Personal Reflections,” OMG Essence Information Day, Berlin, 2013. (http://www.omg.org/news/meetings/tc/berlin-13/special-events/essence-pdfs/S6_Burkhard.pdf)

22. B. Perkens-Golomb, “Successful Utilization of ESSENCE at Munich Re,” Essence-in-Practice Conference in OMG Technical Meeting, Berlin, 2015. (http://www.omg.org/news/meetings/tc/berlin-15/special-events/essence-presentations/perkens-golomb.pdf)

23. E. Seymour, “We All Have Different Ways To Do Things. And That’s OK.” Essence-in-Practice Conference in OMG Technical Meeting, Berlin, 2015. (http://www.omg.org/news/meetings/tc/berlin-15/special-events/essence-presentations/seymour.pdf)

24. C. Zapata and I. Jacobson, “A First Course in Software Engineering Method and Theory,” DYNA, National University of Columbia, Vol. 81, Jan./Feb. 2014. (http://www.scielo.org.co/scielo.php?pid=S0012-73532014000100026&script=sci_arttext)

41