analysis of scientific research mario sangiorgio giordano tamburrelli
Post on 21-Dec-2015
215 views
TRANSCRIPT
![Page 1: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/1.jpg)
Analysis of scientific research
Mario SangiorgioGiordano Tamburrelli
![Page 2: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/2.jpg)
The origin of this work
Carlo Ghezzi’s keynote:Reflections on 40+ years of
software engineering research and beyond: an
insider’s view
Analysis based on papers
Lack of tools to perform the
analysis
WHATresearch topics
WHOcontributors
HOW/WHENtrends
![Page 3: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/3.jpg)
The origin of this work
Time consuming Boring
Requires an expert
Lack of tools to perform the
analysis
![Page 4: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/4.jpg)
Automatic analysis
Faster
ScalableGeneral method
One-click(After
training)
Feasible with data mining techniques
BUTstill not perfect
(it is not semantic-based)
![Page 5: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/5.jpg)
Steps of the analysis
Identificationof subtopics
Interpretation ofpaper content
Trend analysis(So far)
CLUSTERING
CLASSIFICATION
CLUSTERING
STATISTICS
![Page 6: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/6.jpg)
Clustering
![Page 7: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/7.jpg)
ClusteringHierarchical Expectation
Maximization Algorithm
The tool used is Crossbow
Thanks to Gianluca Staffiero and Gabriele Valentini
Abstracts of papers from both general and specificconferences and journals
![Page 8: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/8.jpg)
The clustering process
![Page 9: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/9.jpg)
Classification
![Page 10: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/10.jpg)
Classification
Bayesian classifier
Ad hoc tool using Mallet
Analysis based on the abstract of the papers
![Page 11: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/11.jpg)
Result evaluation
Clustering was iterated until the results were good
Classification performs well:high precision and recall values
human expert agrees with the classifier
![Page 12: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/12.jpg)
Outcomes
Research analysistrends on main
conferences and journals
Tools to support research
automatic bidding
![Page 13: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/13.jpg)
Some trends found
Data from IEEE Transactions on Software Engineering
![Page 14: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/14.jpg)
Some trends found
Data from IEEE Transactions on Software Engineering
![Page 15: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/15.jpg)
Automatic bidding
Build upon analysis methodologies and results
![Page 16: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/16.jpg)
Bidding processGrouping the
submissions by topicCreation of a profilefor the reviewers
Matching papers’ topicwith reviewers’ interests
CLASSIFICATION
CLASSIFICATION
SELECTION
![Page 17: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/17.jpg)
Grouping the submissions
![Page 18: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/18.jpg)
Creation of the reviewer profile
![Page 19: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/19.jpg)
Matching profiles and submissions
![Page 20: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/20.jpg)
Result evaluation
ICSM 2010
![Page 21: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/21.jpg)
Reviewers’ profiles
Carlo GhezziProfile:
web-servicesformal methods
middleware for distributed systemsmodels
software componentseducationCONFI
RMED
Harald GallProfile:
software miningmiddleware for distributed
systemsmodels
empirical studies
Do you agree?
![Page 22: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/22.jpg)
Comparison with actual bids
Results apparently not so good: recall it is about 53%
BUT
The actual bid is not an oracle
We are suggesting papers for the most
relevant topics
![Page 23: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/23.jpg)
Live Testing: ICSE 2011
Propose our bids to the reviewers
Get a feedback on our suggestions, based on reviewer impressions
![Page 24: Analysis of scientific research Mario Sangiorgio Giordano Tamburrelli](https://reader030.vdocuments.us/reader030/viewer/2022032521/56649d585503460f94a37fea/html5/thumbnails/24.jpg)
Future worksImprovement of the system
Ranking of the suggested papers
Deeper statistical analysis
Paper assignment based onGenetic Algorithms assignment