811600s - emerging trends in software engineering · personal experiences, ... • analysis of the...

811600S - Emerging Trendsin Software Engineering

Mika Mäntylä <[email protected]>

Course organization

• Teacher-in-charge: prof. Mika Mäntylä <[email protected]>• Course assistant: Seyedrebvar Hosseini <[email protected]>

Overview of the course• 5 ects

• 5 * (25h-30h) = Workload 133h• Learning Topics and Targets

• Introducing emerging trends in Software Engineering• Why: Course contents (Lectures + articles)

• Writing individual essays of the trends• Why: Internalize the course contents,• Why: Learn to think, compare and reflect based on given material

• Creating and answering multiple choice questions• Why: Creating questions is effective way of learning• Why: Answering questions gives you feedback about your learning

• Practice trend mining from academic and industrial (new!) databases• Why: Allows you to discover the trends on your own• Why: Useful for your Master’s thesis• Why: Learn about academic and industrial databases• Why: Learn about text-mining

• All material and lectures in English• No Exam

Trends selected for the course

• Web-services• Industrial studies of software engineering• Mobile and Cloud computing• Cryptocurrencies• Additional course contents:

• What are trends• Trend mining technology: R + Rstudio + Scopus/Scopus API +

Twitter/TwitterAPI + StackExchange/StackExchangeAPI

Course lectures, exercises, and weekly DLs• Tue: Essay DL (12:00)

• Submit your essay via Optima

• Tue: Deadline for Peerwise questions and answers (23:59)• Deadline for each week answers is one week later than for questions.

• Wed Lecture 12-14• Recap previous lecture, Peerwise questions, Demo of the exercise work and

fundamentals of text mining. Discussion

• Thu Lecture 8-10• New content is presented

• Fri – Exercises 9-12 OR 13-16• Trend mining in computer class room

Lecture locations have variation!

• Exercises in TS135 and YL124

Grading

All exercises must be submitted ontime!

• Start working on the group exercise phases early• Completing the exercises takes time• “The night before the deadline” does not work!

• Individual assignments take a lot of time, too• Few hours is not enough

Passing the course

• Roughly 50% of the points• AND completing tasks: essays and group exercise assignment

All materials see Noppa

• https://noppa.oulu.fi/noppa/kurssi/811600s/luennot

Feedback

• Feedback is welcome in all forms

• Problems? Please, tell us immediately – some thingscan be fixed during the course

• Face-to-face during the lectures• Email to course staff

• The standard course feedback form is published at theend of the course

Now!

• Enroll in WebOodi, now• Read

• The information in the Noppa pages• Exercise and assignment instructions when available

• Announced in Noppa

• Start forming a group of 2-3 students for the exercise work• Talk to other students during the lectures• Group enrolment + topic selection DL: Wed 13.9. lecture

• Be there, if you don’t have an exercise group yet (Mika will help)

Questions?

Peerwise ->• https://peerwise.cs.auckland.ac.nz• For every lecture you can

• create multiple choice questions• answer questions created by other students and the course staff

• Course ID: WILL BE PROVIDED LATER• Identifier: <your student ID>

Peerwise

• Only course staff uses tag: “Practicalities”• You are allowed to link and use extra sources (web-pages, videos) in

your questions• Question requirements

• related to the course educational contents• explanation of the correct answer,• tagged to particular lecture• high quality in general, e.g. professional language• if question has an incorrect answer you have to fix it

• you have the possibility to edit your question after publication

Writing Essay Assignments

Essays in This Course

• are written from the author's personal point of view• present a critical analysis of the topic and source

materials• shall include analysis and synthesis of source materials,

personal experiences, and reflections of the author• must demonstrate understanding of the given source

materials

Essay is an analytical and coherent writing with a clearviewpoint or statement that is logically reasoned andcritically analyzed.

Goal: Help you to learnand demonstrate your learning

• Reflecting on a given topic using source materials– Combine and learn from the sources; find new sources– Use the given questions to guide you– Own insight and comparison of opinions based on the sources is highly valued

in grading• Read the instructions first and then the articles

– Focus to the given viewpoints and questions– Do not try to study all the given material in detail

• Write the essays yourself– Do not copy-paste any text to your essays– Give references and cite properly your sources

• The length of an essay is stated in the instructions– Do not exceed the stated length

• Content is the king– not the length

Own insight and comparison of opinions basedon the sources is highly valued in grading

• What does this mean?

Aids: Own insight and comparison of opinionsbased on the sources

• Point counter-point– Point: “Self-driving cars will completely replace human drivers (source A)– Counter point: “I do not think this is true because the maps are never accurate

enough due to road work. This would make even the smartest pre-programmed machines fail”

– Counter point: “I do not think this is true because the according to Z (that Ifound online/that I learnt about in course X) list 10 things self-driving cars cannever do”

– Counter point: “I do not think this is true because my dad is truck driver andhe says it is not possible”

– Counter point: “I do not think this is true because I have worked on buildingself-driving cars and we problem is detecting anomalies, e.g. flood,construction work etc. So my estimate is that it will take at lease 30 yearsbefore it is even technically possible. Let alone the political and legislativedifficulties which…”

• Think how is this information useful– I can use it “in my Master’s thesis” / “when I start my own company

developing” / “when I work as a programmer in XXX”

Aids cont’d – Answer good question in your essay

• Facts / Memory (25%)– Content that can be Googled in minutes– Who, Where, What, When– When was Agile Software Development

introduced– What is Model-Based testing


• Convergent thinking – Analysis and integrationof information (25%-75%)– Why, How, In What ways– Why was Agile Software Development introduced– How does Model-Based testing differ from regular

test automation


• Divergent thinking – Own ideas new directionson a given topic (0-50%)– Imagine…, Suppose…, If… Then…, What are some

possible consequences– What are some possible consequences of

introducing Model-based test automation toorganization performing currently manual testing?

– How might Agile Software Development change inthe future?


• Evaluative thinking – Judgement, Value,Choice, Own opinions (0-50%)– Defend, justify, What do you think about– What do you think are the benefits of Agile

Software Development compared to Plan-drivendevelopment?

– How do you feel about the goal of 100% testautomation?

Essay is NOT

• just a summary of the material– ”referaatti” in Finnish

• a random list of issues and findings related tothe topic

• direct answers to a list of questions

Basic Structure

• Introduction– State the facts

• Question / Position statement, or Viewpoint• Discussion

– Analyze the facts

• Conclusion

Length

• Remember: Content is the king!• The length of an essay is 1200-2000 words in

English and 923-1540 words for Finnish (Onlythe body text is counted).The Finnish languageuses less words than English for identicaltexts. This is documented in translation fees ofpeople see here and here

Common Pitfalls• Writing only summarizes the material, writer does not take own viewpoint or position on the topic.

– Essay covers the topics of different source materials separately and does not combine.– Contents of the source articles were referred superficially without demonstration of understanding of the authors'

ideas in the context of the essay topic.

• Organization of the essay follows strictly the questions in the assignment and does not form a coherentstructure.

– introduction/motivation, findings/discussion, conclusion

• Source material is not used and learning the core concepts is not demonstrated– Mainly own ideas presented without reflecting to the materials

• Argumentation and reasoning of the writers opinions and viewpoints are weak.– Bold claims and firm statements without references or logical reasoning.

• Lack of critical analysis– Writer does not demonstrate critical thinking when covering the source materials.

• Don’t repeat your ideas too much – it is easy to write two paragraphs that essentially tell the same storywith different words.

– Read your own text with critical eye, what are you actually trying to say in each part of your essay.

• Language and style:– Proofread your text before submitting to fix obvious typos, broken sentences and incomprehensible paragraphs.– Cite your references accurately in the body text where you use the materials, e.g., citing this slide set (Mäntylä 2015).

Do not put citations at the end of the paragraphs. Use the instructed citation style (Anonymous 2014).

See Examples

Emerging Trends inSoftware Engineering

Exercices

811600S

Goal of the Exercices

• Write a ”Citation & Trend analysis”, document of a selected topic…• statistical analysis of papers/tweets/questions• analysis of the trends in the topics in a specific field of software engineering

• Data collected from Scopus/Twitter/StackExchange,• If you use single source. Max possible grade 70% (=3)

Topic for Trends Analysis• There are no pre-given topics• You should start thinking about your topic now• No two groups should have the same topic!• It is possible to have overlapping topics, e.g.

• Group 1, Topic: “Test Automation”• Group 2, Topic: “Android test automation”• Group 3, Topic: “Selenium”• Group 4, Topic: “JUnit”• Group 5, Topic: “Appium”• Group 6, Topic: “Performance test automation”

Expected results for submission

Please zip the following (final) material set for submission…• A bibliometrics assessment document (word-document)• Scopus/Twitter/StackExchange data (original csv-files & Excel graph-

files)• R-script files→ only if other than example scripts used• Rdata-files = R workspace, R history data, generated data files.

To be returned as zipped, by e-mail.

Schedule• Exercises – same time & place each time

• Week 1:• Form a group,• Decide on a topic (you can also form a group based on common interest• Formulating a search string. Use web intefaces to test results• Validate a search string

• E:g. if one searches for ”Selenium” wo ”software” then we get a lot of bad results• Export data• Start statistical analysis

• Week 2:• Statistical analysis (continues)

• Week 3:• Trend analysis with R Start topic analysis with R: 5 most common / hot & cold topics

• Week 4:• Continue with statistical & trend analysis

• Week 5:• Start writing and reposting

• Week 6• Finalize papers

Way of Working

• Excercises done in classroom• Assistant to help and follow your progress

Immediate actions

• Note - for TS135 you need user account: TS494/Virtanen or TS495/Mäkinen

• If you use your own laptop è please install the required sw inadvance!

• Form groups of 1-3 persons• Select a topic

In the execercises you will…

• Familiarize yourself with Scopus/Twitter/StackExchange and data inthe selected forum

• Export data from Scopus/Twitter/StackExchange – get all data at once(db updated regularly)

• Analyze the data using Excel & R/RStudio• Write a bibliometrics assessment document based on the analyzed

data (~20 pages)• Deadline to be defined.

More details will be given in theexercise sessions and lectures(Wednesday)

Example research questions to cover

• How many publications/tweets/questions there has been annually?• What is the distribution of citations/retweets/upvotes these

publications/tweets/questions (yearly comparison)?• How many publications/tweets/questions have been cited?• How many publications/tweets/questions have not been cited, what is the ratio?• What are the most cited publications/tweets/questions?• How has the evolution of citations/retweets/upvotes varied over the period of

time?• How is the contribution of publications/tweets/questions?• Have the focus areas of the publications/tweets/questions changed over the

years? If yes, how?• Is there observable overall impact trend within publication topics?

Example Table of Contents• Introduction• Goal and research questions• Data source & methods• Results

• Volume of Papers• Distribution of citations/retweets/upvotes (number of citations/retweets/upvotes vs number of documents)• Top 20 cited publications/tweets/questions• Evolution of citations/retweets/upvotes over period of time• Observed differences• Focus areas with interactive cluster exploration (LDAVIS)• Focus areas of the publications/tweets/questions

• Finding the optimum number of topics• Hot and cold topics• A review of one topic: xxx

• Top 5 most popular and cited topics• Potential threats of validity• Conclusions• References

Scopus/Twitter/StackExchange

• Scopus - The largest abstract & citation database of peer-reviewedliterature

• NOTE - Available via credentials & network provided by the University

• Stackexchange – Expert question answer site• We will use StackOverflow that specializes in software related questions

• Twitter – Popular social media site

Rstudio – Analyzing the data

• R is a language and environment for statistical computing andgraphics.

• RStudio IDE is a powerful and productive user interface for R.• The intention is to use R/Rstudio to analyze the data with available

scripts.• The language itself is not studied in detail.• For those interested, there is a lot of material and self-learning

tutorials available in the web, e.g. An introduction to R


• The data acquired from Scopus/Twitter/StackExchange (csv-file) willbe used as input.

• The scripts are available.• Define the working directory to the script-files!• Citation/Retweet/Upvote data from Scopus/Twitter/StackExchange

needs to be readable for the tool• No empty fields (e.g. [No author name available] or [No abstract available])• Correct field separator character• No spaces in the data headers (e.g. change “Cited by” to “CitedBy”)


Note - R requires additional packages to be added for the analysis. Thedefault location is the ProgramFiles –directory – to which students donot have rights (admin rights required to write to the location).To avoid the problem, use personal directory, e.g.

# install.packages(“tm”, lib=“C:/users/<username>/<directory>”, dependencies = TRUE)install.packages(“tm”, lib=“C:/users/mickeymouse/Documents/RPackages”, dependencies = TRUE)

# library(“tm”, lib.loc=“C:/users/<username>/<directory>”)library(“tm”, lib.loc=“C:/users/mickeymouse/Documents/RPackages”)

Rstudio – Analyzing the dataSeveral panels …

Top Left: Syntax-highlighting editor forsource code

Bottom Left: R Console

Top Right: Environment mgmt / dataview

Bottom Right: File browser / plotwindow / package install window / Rhelp

• Script files are opened by selecting: ”File” => ”Open file…” and browsing to find the correct R-file.

• The commands can be run by placing the cursor on the source row and…

- Clicking ”Run”… or

- Pressing ”ctrl” + ”enter”

- Note - several commands (rows) can be selected and run at once.

• A command may be written to the console and run by hitting ”enter”.

• Environment data & history can be examined.

• Workspace can be saved to be able to continue with the same data.

• To open an existing workspace, (.Rdata): ”Session” => ”Load Workspace…”
