using developer conversations to resolve uncertainty in software development: a position paper

PowerPoint Presentation

Ahmed Shah Mashiyat, Michalis Famelis, Rick Salay, Marsha Chechik

University of Toronto

RSSE@ICSE 2014Using Developer Conversations to Resolve Uncertainty in Software Development: A Position Paper

Software Development is Social2

Conversation between developers on jazz.net about an enhancement ticket


Conversation between developers on jazz.net about an enhancement ticket

Ideal conversation: Question, Debate, SolutionThe Managers PerspectiveGoals:Product qualitySpeedy delivery

So, conversations should:cover all issuesinclude all stakeholdersnot take longer than necessary

But that is not always the case4

Manager


(The conversation has been abstracted and simplified for presentation purposes. The listing to the left is not intended to be legible.Software Development is Social6

Ali is leading the development of a component.(The conversation has been abstracted and simplified for presentation purposes. The listing to the left is not intended to be legible.


He seeks input from stakeholders.`Color?

Orientation?


Stakeholders start discussing.

?

?

Christine



?

?

Bao



?

?


Ali thinks he has found the answers he needs

=

=

Bao agrees


Ali marks the issue as resolved

=

=


=

=

But Dimitra thinks that the issue is not settledSize??


=

=

But nobody takes her objection into accountSize??

Guys?Conversations are MessyThe decision about orientation was not debated

The question about size was left unanswered

Stakeholder Dimitra was not consulted until late

Changing the status to resolved was premature

15

Manager

Proposed Solution16Mine conversationsGather AnalyticsGenerate Action Recommendations


Generating RecommendationsUse metrics to support managers in facilitating conversations:

18

Intervene when questions are left unanswered for a long period of time

Alert when questions are prematurely marked as resolved

Report when a question has been discussed enough to be considered mature

If a question is re-opened, create a context reminder of previously examined alternativesProposed Solution19Mine conversationsGather AnalyticsGenerate Action Recommendations

Online vs Offline (post-hoc)

Main challenge: determining resolution of a point of uncertainty:Trusting status changes not necessarily a good ideaWho should be part of discussion?Peoples stance may not be obviousE.g. does elaboration imply endorsement?What does absence of comment (a persons default) mean?

For now: Compute metrics from the structure of the conversation20Gathering Analytics about Uncertainty

Dashboard Mock-ups21


Mining the Conversations23

`

Identify unique Points of Uncertainty from question utterancesMining the Conversations24

Identify unique Points of Uncertainty from question utterances

Associate them with Proposed AlternativesMining the Conversations25

Identify unique Points of Uncertainty from question utterances

Associate them with Proposed Alternatives

Track the stakeholders arguments

Mining the ConversationsIdentify unique Points of Uncertainty from question utterances

Associate them with Proposed Alternatives

Track the stakeholders arguments

Recognize the Resolution of Uncertainty26

Construct a Model of the Conversation27

Twente Argumentation Schema (TAS)

Time: left-to-right, top-to-bottomConstructing the TAS Model28Automation Challenges

Identifying unique points of uncertainty among many utterances of questions

Identifying proposals when they are not stated explicitlye.g. cultural differences in assertiveness

Mapping utterances of proposals to points of uncertainty

Benefits

Associates points of uncertainty with proposed answers

Easy to compute structural metrics

Can be used to trace uncertainty to software artifacts

Summary

29

qualityspeedFuture WorkHow to best automate the creation of the TAS model?

How can we safely determine that a point of uncertainty has been resolved?

What are the most useful metrics, analytics and action recommendation for managers?

How to connect uncertainty in conversations with relevant organizational, social and software models?30Questions?

icons by:31

using developer conversations to resolve uncertainty in software development: a position paper

Technology