supporting dynamic communication by software development histories

Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University

Supporting Dynamic Communication by Software Development Histories

Katsuro Inoue

Makoto Matsushita

Kei Sasaki

Osaka University

Nov/08/2004 Dync Symposium 2004

2Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University

Personal activity data Problem detectionTool customisation/activity navigation

Single project data Progress estimationProcess/product improvement

scale

Collection Analysis Improvement

4

23

65

Multiple project dataProject comparison/expertise extraction

Organizational assetreuse

Classification of SE Technologies by Target Scale

MegaSoftware

Engineering



Mega Software Engineering MSE

Targets many projects

A new concept but not a new technology itself

Collection of key technologies already existing and emergingDistributed environment and data sharing

Analysis and data mining

Project monitoring and controlling

Scalable computing

...

Use advances of hardware performance, e.g., network bandwidth, CPU clock, memory space, disk capacity, ...

Software engineering technology should share in advances of hardware, which is mainly used for multimedia, grid, simulator, ...



Characteristics of MSE

Experience and knowledge of individual developer or project are collected, refined as assets, and reused in community

Non-structured flat dynamic community for information sharing

View from the organizational benefits may be directly obtained

Open source community is an example case of MSE

Concept of Empirical Project Monitor EPM

Analysis

Software Development Organization

Related Organization

　　　

　

　　　　　　

Internet Public Domain SoftwareOpen Source Project

Collection

Improvement

Versioning(CVS)

Mailing(Mailman)

Issuetracking

(GNATS)

Other tooldata

FormatTranslator

FormatTranslator

FormatTranslator

FormatTranslator

Process data archive(XML format)

Product data archive(CVS format)

Code clonedetection

Componentsearch

Metricsmeasurement

Projectcategorization

Cooperativefiltering

CorporateSource

GUI

Managers

Developers

Project xProject yProject z

. . .

EPM（ developing）



S P A R S-J

Internet / Corporate Repositories

Software Component Searcher

ComponentCollector

Analyzer and Evaluator

ComponentArchive

Query Handler

Software Product Archiving, Analyzing and Retrieving System for Java

SPARS-J

CoxR

CoxR (Web Server)

CoxR user

Query WordQuery Word = Source code Commit log Keyword, Time File name Developer name

Search Result

CoDS

Related Files /DataCVS information

E-mail information

Developer nameTimeTopics

Commit log Keyword

TimeFile name

Developer name

Data Display Record System

Search Result

File nameDeveloper name

Time

SPxRCode DB

DB Create tool

Fusion info Create tool

CVS Info DB Fusion info DB E-mail Info DB

CVS info Create tool E-mail info Create tool

Token

SimilarityTimeFile nameDeveloper name

Source code

Token compare tool

CGI-MainSource code Search result

Lexical analysis tool

CVS Repository E-mail Archive

Code DB

DB Create tool

Fusion info Create tool

CVS Info DB Fusion info DB E-mail Info DB

CVS info Create tool E-mail info Create tool

Token compare tool

CGI-Main

Lexical analysis tool

Data Display Record System



Extension of CoxR to Fit to DynC

Unveil development community by repositories

• Extracting knowledge (5W1H) from SCM repository, email archives, bug tracking system

• Detecting relationship among persons, products, email, …

• Find communities according to the network of relationship.



Classify KnowledgeKnowledge: intention + task

Development policy

Requests for changesNew features

Intention… of software development

Discussion of whole system architecture

Problem with a new code

TaskResults of development

Features enhanced

Actual software changes

Simple Example

Bug type:

software

way to happen: ifconfig lo0 192.168.2.1 alias ipfw 50 add fwd 192.168.2.1,23 tcp from any to 192.168.2.1 some time ...

description: Error….

Kernel panic happened when I changed

ipfw(8) rules, why?

administrator

No more panic even if syncache_drop() is called just aftter sync

ache_respond().

src/sys/netinet/tcp_syncache.c 1.5.8

Bug report on the kernel panic2003/05/22

Bug fix2003/06/14

This problem is not related to ipfw(8); see previous email threads

for how to fix.

On certain situation, a race condition will happen while tcp_syncache() calls outside

functions.

using memory after freed in tcp_syncache ・・・・2003/05/28

Patch needed to src/sys/netinet/tcp_syncache.c and tcp_var.h, however, it may cause unwanted side-effect to others.

src/sys/netinet/tcp_var.h 1.56.2.12

Development history

Bug tracking system Mailing list CVS

Discussion B

Discussion A

Workaround

Bug overview

Knowledge Map to Intention/TaskDon’t panic even if syncache_drop() is called just aftter syncac

he_respond().Type of bug:Software-bug

How to reproduce: ifconfig lo0 192.168.2.1 alias … Description: ….

This problem is not related to ipfw(8); see

previous email threads for how to fix.

On certain situation, a race condition will happen while tcp_syncache() calls outside functions.


On certain situation, a race condition will happen while tcp_syncache() calls outside functions.


Don’t panic even if syncache_drop() is called just aftter syncac

he_respond().

This problem is not related to ipfw(8); see

previous email threads for how to fix.

Kernel panic was happen when I

changed ipfw(8) rules, why?

Type of bug:

Software-bug

How to reproduce: ifconfig lo0 192.168.2.1 alias …

Description: ….

Intention： Development directions, problems, and reasons

Source code (diffs)

Task：What’s happened, actual code changes

Kernel panic happened when I changed

ipfw(8) rules, why?



Contents of Intention/Task

FilenameRevisionTag, Date

KeywordsFilenameRevision

FilenameBTS IDSeverityStatus

Description

BTS

E-mail

CVS Log message

SubjectMail body

Submit dateCategoryBug typeOverviewHow to repeat

Intention

Source code•Inserted•Deleted

How to fixLast updated dateUpdate historyClosed date

Task



Intention/Task Networks

Feature A

BTS a

Log d

E-mail c

CVS α

CVS θBTSλ

CVS β

BTS ζCVS γ

How do I know how to fix the failure in feature A? It seems that that files may

have a bug…Task Set B: modify similar files

E-mail g

BTS f

E-mail e

Task Set C: similar features of ACVS ψ

CVS ωCVS ν

CVS δ

Task Set D: similar contents

intention set of similar bugs

intention set of feature BSimilar modifications to files

Similar BTS entries

debugger

E-mail b

Extract intention/Task Sets, whichare related to current topic

CVS φ



Process DB

System Overview

Web Server

Intention and Task Developer (Group),Role

E-mail

CVS

CoxR

Supporting DC System

Role Develop Group

Source codekeywords

LogFilename

Developer name

Query Word

Task DB Intention DBRole DB

results

Query Word

Results

System Controlintention

task

user

Knowledge

Intention DBRole DB

Analyzer

System Control

Rel. DBRel. DB

Bug Tracking System

Relations

情報抽出

Extractinfo

Task DB

Query Word = Source code Commit log Keyword, Time File name Developer name



Conclusion

Mega Software EngineeringEASE, SPRAS-J, CoxR

DynC as a part of Mega Software Engineering

Extensions of CoxR

Knowledege: Intentions and TasksExtract and Grouping

Developers Role

Extracting dynamic community from the network organized with intentions and tasks

System is under implementation



IntentionLog message

It tells us about “hat kind of changes has been made, or why this change is needed, toward the files committed at the same time.

Subject・mail bodyDiscussions between developers on future directions, a certain source code, how-to-fix the bug.

BTS entry

The type of bug, how-to-repeat the bug, what failures are there, what change is desired, are described clearly.



Task

Source codeActual changes to software itself

Easy to know what portion was changed, by checking the delta against previous revision.

Fix section (in each BTS entry)It records modification histories and what changes will fix the bug.



Relationship between developers1. Detect core development group of the software

Developers who join similar intention SetsJoining many discussions

Developers who join similar Task SetsDeveloper who joins the same products

2. Define roles of developers

Total programmer

Programmer

Tester

Debugger

Modify many portions of software

Modify several portions of software

Submit many change requests

Manager Controlling project itself

Mainly fixing bugs

Administrator Managing stable products



Grouping intention/Task

log

Task

Task

Task

How-torepeat

log

Same revision

Information with a same filename, revision, BTS ID

Same BTS entry

code

fixes

fixes

code

E-mail

Intention

Intention

codeWork-around

Information in a same file

Modification according to workaround

Discussion on fixes

How-torepeat

Discussion on architecture

Modification request



Relationships between intention/Task

Each intention/task is spread across some systems, such as CVS, email, etc.Also, relation between each intention/task is also scattered about repositories.

logIntention

Task

code

fixes

Intention Set Task Set

May contain:•Source code•Bug fixes

May contain:•Development log•Email•BTS entry

TaskEmail

BTS



Extracting developers concerned

E-mail b BTS a

Log d

E-mail c

CVS α

CVS θBTSλ

CVS β

BTS ζCVS γ

Ok, maybe that’s all information I may

consult, but where to check first?

E-mail g

BTS f

E-mail e


CVS ωCVS ν

CVS δ



intention set of feature B

Task Set B: modify similar files

Main feature developersDevelopers ofmanaging bugfixes

I’d like to know who implemented the feature, and who fixed some bugs

CVS φ



Narrowing networks down

E-mail b BTS a

Log d

E-mail c

CVS α

CVS θBTSλ

CVS β

BTS ζCVS γ

E-mail g

BTS f

E-mail e


CVS ωCVS ν

CVS δ



intention set of feature B

Task Set B: modify similar files

I’d like to check similar features’ development histories and bugfixes

…

CVS φ



Find what you need on your concerns

E-mail b BTS a

Log d

E-mail c CVS γ


CVS φCVS φCVS φ’

CVS rCVS rCVS rCVS γ’

BTS p

BTS k

E-mail h

Past development of the feature

Further policies on developing the feature

Bugfix information of the feature

CVS φ

BTS ζ

Task Set C: similar features of AAfter identifying the developers and knowledge automatically,find what you want by traversing information selected.

Development results from the past



Gathering the answer

E-mail b BTS a

Log d

E-mail c

CVS φ

CVS γ

intention set of similar bugs CVS φ

CVS γ

CVS φCVS φCVS φ’

CVS φCVS φCVS φCVS γ’

BTS p

BTS k

E-mail h

CVS φ’

CVS γ’CVS γ” E-mail h

E-mail cBTS ζ

Task Set C: similar features of AGathering the “how-to fix” appropriately,from the Task Set of similar feature



Data extractionExtracting intentions and Tasks

Data analysisIntention analysis

Grouping Intention SetsTask analysis

Grouping Task SetsData integration

Relationship among Sets

System implementation

Database managing unit

Role AnalysisCore development

group and roles

Create DBs

Data extractionExtracting intentions and Tasks

supporting dynamic communication by software development histories

Documents