t-76.115 project review

T-76.115 Project Review

RoadMappersFinal demo

6.4.2004

2

T-76.115 Project ReviewT-76.115 Project Review

Agenda

Project background (5 min) Demo (15 min) Delivery phase (3 min)

Achieving the goals of the iteration Realization of tasks

Project in general (15 min) Project progress (PP-I3) Aftermath Results Used work practices and tools

Comments and questions (5 min)

3


Background

Main task “Create a graphical user interface for a modelling tool for configuration tasks”

Other groups (Confuse and Comet) in previous courses have produced the underlying system

The Sarcous research project is studying the possibility to model software product families and their components

Aim to bring more automation in the configuration process The tool will be used in the research of software product families

Testing different modelling methods for the variability in software product families

Tool provides a interface for drawing UML-like diagrams of product families and assigning dependencies like requires and conflicts to the components in diagram

4


Demo

Some of the key functionality will be presented during demo

5


DE: Goals

Fix remaining defects and finalize system Some minor bugs remain They are documented

Finalize project documentation - OK Delivery of system – Customer has received system Evaluate own work – Final report Hold final meeting - OK

6


DE: Realization of the tasks

Bug fixing took more time than first estimated

Some new bugs found Estimate was updated during phase

More time in testing and defect reporting also

Acceptance testing did not require effort

name done plan diff left

*TOTAL* 203.00 189 14 0.00

GE: Infra 4.00 3 1 0.00

GE: Internal communication 4.00 5 -1 0.00

GE: Meetings (Internal) 12.00 14 -2 0.00

IM: Update User manual 7.00 4 3 0.00

PM: Final review and preparation

9.00 12 -3 0.00

PM: Meetings (Customer) 3.50 10 -6.5 0.00

PM: Other project management 5.50 5 .5 0.00

PM: Personal SE practice 15.50 21 -5.5 0.00

PM: Write final report 22.50 20 2.5 0.00

TE: Defect reporting 10.00 5 5 0.00

TE: Fixing defects 95.50 80 15.5 0.00

TE: Regression testing 9.50 5 4.5 0.00

TE: Test reporting 5.00 5 0 0.00

name plan

*TOTAL* 5

TE: Acceptance testing

5

name realized plan

DS: UI design 5.00 17

Unplanned 1.00 0

Realized tasks

Not started Unplanned

7


Project progress – Project planning

First meetings Requirements definition

Done by requirements team Elicitation, analysis, documentation and validation phases

Project planning Goals were set

Learning goals for team Top-10 goals of customer

Work practices -> Quality Handbook Project infrastructure was created

Server, CVS etc.

8


Project progress – Implementation 1

Work practices were clarified Design of UI in general and some use cases

UI prototype was created Heuristic evaluation Usability tests

Design of core architecture Start of implementation was delayed

Not enough time to implement GUI-module Design decisions were demonstrated with the prototype Prototype was later used as base for GUI-module

Module integration took more time than expected Not enough time to test Testing was done in the beginning of the next iteration

9



Testing of the previous iterations implementation Design of remaining use cases

Planned usability tests were moved to I3 Heuristic evaluation was done

Architecture was refined Remaining use cases were implemented GUI-module was started

Functionality existed in the underlying code, but GUI did not offer support for everything

Some of the system test could not be run The phase was divided by the exam season and Christmas holidays

Had some trouble in getting back to work, but managed to catch up the planned schedule

10



Last bits of design Pop-ups, icons etc. Usability tests Heuristic evaluation

Remaining parts of GUI-module were implemented Rest of the iteration consisted of testing and debugging Unit testing was given less attention due time constraints Peer testing

Peer group tested our system System was not mature enough at this phase Only six new bugs found

We tested peer groups system

11


Project progress - Delivery

Mainly testing and debugging Acceptance testing done by customer Final meeting was held Final version with some bug fixes delivered to customer after acceptance

testing Document delivery

Final report Final versions of other documents

12


Working hours by person in DE

The implementers (Enbuska, Siltanen, Latto) still had quite a lot of work with the bugs in the last iteration

Others could not participate in this Also more work for test manager

Real Plan DiffPohjola 27,5 48 -20,5Enbuska 36,5 33 3,5Saastamoinen 36,5 34 2,5Sarmanne 28 24 4Siltanen 28,5 18 10,5Karkkunen 24 29 -5Latto 22 8 14Total 203 194 9

Realized hours in this iteration

13


Working hours by person in the whole project

As whole the project stayed on budget Deviation was 1,5 % (22 hours)

Individual deviations were between -9% (18,8 hours) and +8% (16 hours) The roles were clearly divided and in the end it was very difficult to try to

participate in other persons work load BUT not all the effort was marked to Trapoli

For example writing an e-mail takes about 5 minutes -> no need to report this What if you wrote 100 e-mails during this project? Over 8 hours of work that was never reported We tried to correct this by creating a task for internal communication Other similar activities (hidden work) still exist

Pohjola Karkkunen Siltanen Enbuska Sarmanne

Latto Saastamoinen Total

PP 66,0 49,0 30,0 34,0 37,0 54,5 44,0 314,5

I1 30,0 53,0 48,0 64,0 56,0 70,5 45,0 366,5

I2 24,5 32,0 51,5 30,0 41,5 33,5 34,0 247,0

I3 33,5 39,0 54,5 40,0 43,5 35,5 45,0 291,0

DE 27,5 24,0 28,5 36,5 28,0 22,0 36,5 203,0

Total 181,5 197,0 212,5 204,5 206,0 216,0 204,5 1 422,0

14


Quality metrics 1/2

Two usability tests (I1 and I3) First with two users Second with four

Heuristic evaluation Three analyses, five evaluators Majority of problems found were minor problems Proposals for improvement and reporting the problems were most time-

consuming Valuable results with relatively small effort

Catastrophic Major Minor Cosmetic Not a problem TotalUsability tests 0 7 13 6 2 28Heuristic evaluations

0 8 22 13 10 50

Usability problems

15


Quality metrics 2/2

Metric I1 I2 I3 DETime used for testing 3 24 12,5 9,5Time used for fixing defects 0 10,5 55 95,5Total number of test cases 0 64 81 81Number of test cases run 0 35 71 80Number of test cases passed 0 8 36 76Number of defects found 0 36 65 60Number of fixed defects 0 36 62 50Number of verified fixes 0 36 62 50

C S M TotalController 1 0 0 1

GUI 19 74 48 141

Model 4 2 1 7

Views 6 6 0 12

Total 30 82 49 161

Unfixed bugs 13 1 critical 5 serious 7 minor

Unit tests: No new tests made Old ones slightly updated Two test cases removed All 53 test cases pass

16


Quality assessment

Final product satisfies item pass/fail criteria for unit level and acceptance level

It fails on system level, because there are critical and serious issues that are not resolved

Test case TC-3-13 could not be executed, because it requires that the endpoints of the relationships can be dragged with mouse

This also makes test cases TC-3-11 and TC-3-12 fail.

Functional area Coverage Quality Comments

Controller 3 Only one reported defect

GUI 2 Most of the defects here. No unit tests.

Model 2 Nothing serious

Views 3 Nothing serious

Legend

Coverage:

0 = zero coverage

1 = poor coverage

2 = adequate coverage

3 = good coverage

4 = excellent coverage

5 = full coverage

Quality:

= quality is good

= not sure

= quality is bad

17


Software size in Lines of Code (LOC)

PP I1 I2 I3 DETotal – Controller 0 1213 2096 2662 3114Com – Controller 0 315 688 883 1006Total – GUI 0 1586 3782 7757 13245Com – GUI 0 857 277 1614 2443Total – Model 0 1221 5855 6787 7944Com – Model 0 585 2721 3060 3382Total – Views 0 4020 3362 3458 4497Com - Views 0 1757 1341 1281 1725Total 0 4020 14818 22281 28855Comments 0 1757 4750 6838 8587

GUI contains generated code Total LOC/COM without GUI 9465/6144

For comparison Comet data LOC 8987 parts used by us 5858 COM 3466 parts used by us 2626 LOC+COM 12 553 parts used by us 8484

Our underlying code is bigger than the whole Comet code

18


Project goals

Receive a graphical configuration modeling tool Customer has received the tool

Be able to edit a configuration model (change type-hierarchy and part-hierarchy, edit types etc.)

Be able to view only part of the model Create constraints between different types Create ports and connect ports to each other

Connecting ports does not work, but is not necessary either Good communication with project

Customer is satisfied Stable program, no unexpected crashes Traceability of code to specification and to requirements

Customer did not have time to verify this Group has no experience in this either

Architecturally good product, i.e. module divisions, clean interfaces Customer is satisfied

For the project team to learn why 6,7,8 and 9 are important

19


Risks

Risk management utilized Riskit method (J. Kontio) Produced risk analysis efficiently Required active monitoring and controlling actions

24 risk were identified and analyzed during the project Most of these during the PP iteration

11 risk materialized Risk rankings: 1-5, 8, 9, 12, 13, 15, 19 2 risks materialized 3 times 5 risks materialized 2 times

Rankings of the materialized risks are high This indicates that risk analysis was successful

Utility losses of the materialized risks decreased a little during the project Risk monitoring and controlling were quite effective

20


Work practices 1/2

Personal assignments Project progress tracking and control -Eila Configuration management - Tuomo Refactoring- Samuel Heuristic evaluation - Sanna Usability testing - Marko Design patterns - Jan Automated unit testing - Mikko

Hour reporting Some problems in the beginning Trapoli was unstable

Communication practices Internal meetings improved during projects E-mail was major communication tool Talk was used in urgent situations

Risk management Requirements management

Definition done in PP phase Some obsolete requirements, but no new

21


Work practices 2/2

Architectural design Done by a team Design patterns used

Coding conventions Made code readable Some deviations occasionally

Document reviews First with the whole team - Inefficient Later only few members participated in reviews when needed

Testing Unit and system testing Test manager in charge of things

Bug reporting Done with Bugzilla – usability problems Time consuming

Peer testing Very little use But nice experience

22


Questions?

t-76.115 project review

Documents