Annual ProgressSeminar

1st

1st step towards doctoral research

Under supervision ofProf. Sridhar Iyer

Prof. Aniruddha Joshi

Rwitajit Majumdar

Teaching-Learning of Visual Analytics

JAN feb mar apr may jun julyaug sep oct nov dec

JAN

ET 802 Research ProjectCredit…………………RM

ETS801 Seminar: Hand skills Teaching & Learning CreditID405 Human Computer Interaction

sit through…………HCI

ET 801 Introduction to Educational TechnologyCredit………………..Intro ETHS 699 Communication and Presentation skill Non-Credit………..HSSID 665 Craft, Creativity and Post Modernism Credit..................CCIN 609 Visual Design for Interactive Systems sit through………..VD

CS 101 Engagement study

LAMP – Large Scale Addressal of Muddy PointsPULSE – Protocol oriented Utility for Logging Student Engagement

Advance topics in Cognition

Research Question

Methodology• Sample• Design• Variables• Analysis

Results• Compute• Represent

Interpret

Generic problem in research

Research Question

Methodology• Sample• Design• Variables• Analysis

Results• Compute• Represent

Interpret

1312 words extracted from the titles of 165 research article published in Computers & Education in 2013

http://timc.idv.tw/wordcloud/

1. What are the topics?2. What did the figures &

tables convey?

Educational environments and technology

Distance education and learning

Multimedia Education

Education and performance

Teacher Education

design & development

systemic change

research & theory

others

0 10 20 30 40 50 60 70

58

1

22

18

2

23

1

38

2

Kucuk, S., Aydemir, M., Yildirim, G., Arpacik, O., Goktas, Y., “Educational technology research trends in Turkey from 1990 to 2011” Computers & Education 68 (2013) 42–50

N=165

Research Question

Methodology• Sample• Design• Variables• Analysis

Results• Compute• Represent

Interpret

Research Question

Methodology• Sample• Design• Variables• Analysis

Results• Compute• Represent

Interpret

133

figures tables

7 22 3

demographics

statistics

description and example

specific result

lit review

0.00 50.00 100.00 150.00 200.00 250.00 300.00 350.00 400.00 450.00 500.00

51.00

445.00

122.00

132.00

20.00

Distribution of Tables

bar; 65scatter plot; 11

line; 33pi and

venn; 7

model; 107

specific diagrams / fig; 166

screen shots; 176

photo-graphs; 42

others; 21

Distribution of Figures

C-mapOntologyHistogramCircleBox plot

What am I trying to establish?

leverageProper usage of Tables and Figures evidence to support claims in the study

Different studies use different graphs scopeas designer: evaluate

as trainer: implications on teaching-learning

Different tools have different affordances

Difference in interpretation?

Educational Technology Research

Student Engagement1 Student Learning2

Technology Enabled Learning Metric

Accessibility

Effectiveness

Attractiveness

EfficiencyET

1. Aditi Kothiyal, Rwitajit Majumdar, Sahana Murthy and Sridhar Iyer. Effect of think-pair-share in a large CS1 class: 83% sustained engagement. ACM Intl Computing Education Research Workshop (ICER), San Diego, USA, August 2013.

2. Shitanshu Mishra and Sridhar Iyer. Problem Posing Exercies (PPE): An instructional strategy for learning of complex material in introductory programming courses. IEEE Intnl Conf on Technology for Education (T4E), Kharagpur, India, Dec 2013.

Behaviors

How it changed?Problem posing skillHow does it affect learning?

Student Engagement1 Student Learning2

1. Aditi Kothiyal, Rwitajit Majumdar, Sahana Murthy and Sridhar Iyer. Effect of think-pair-share in a large CS1 class: 83% sustained engagement. ACM Intl Computing Education Research Workshop (ICER), San Diego, USA, August 2013.

2. Shitanshu Mishra and Sridhar Iyer. Problem Posing Exercies (PPE): An instructional strategy for learning of complex material in introductory programming courses. IEEE Intnl Conf on Technology for Education (T4E), Kharagpur, India, Dec 2013.

N 450

Batch of 2013

Classroom Observations during Think-Pair-Share activity: Behaviors

Pre test – Post testUnstructured interviews

Observed student’s behavior

The difference in the distribution of Pre-Test

vsPost-Test

was not statistically significant

But, the unstructured interview had evidence that the problem posing activity was engaging, non-trivial and interesting to work out.

In order to understand the dynamics further nature of problem created and the Pre-Post test performance

was studied

Often the process of interaction is studied but the tracking across the process is not done.

Polarized operations:Aggregate statistics evens out the rich variation in dataTracking individual parameter and explicating trends are often difficult to

structure

Requires1. A structure for analysis2. Representation to explicate patterns in the data

Some approach that can group the sample according to set criteria, that the researcher focus to study,

and study their migration

What exists:

Sophisticated Time-series and Cluster Analysis.

else

• Researchers calculates distribution for each phase of tracking • Represent it through pie/bar chart• Calculates how the sample dynamics change on certain parameter

• Writes an elaborate paragraph to explain trends.

SAT Diagram is a unified graph

representing distribution of stratified categories

based on attributes of collected data as its nodes,

which are then tracked along different phases of

any activity for a given sample.

Between each phase it is a complete bipartite graph.

Stratified Attribute Tracking Diagram

Activity: The event for which the data is logged.

Phase: The different parts of the activities that is analysed.

Strata: Group formed out of the sample based on the predefined criteria of the attribute value.

Tr(imi+1n) indicates the t-ratio between group ‘m’ in phase ‘i’ group ‘n’ in phase ‘i+1’.

It is calculated as the ratio of the sample size that migrates between phase-i-group-m and phase-i+1-group-n to the initial sample size of phase-i-group-m.

Stratum distribution: of a certain group in a phase is the ratio of number of people in that group to the sample population.

No. Study   Phase 1 Phase 2  Phase 3 

1 Quantifying Student Engagement

Name Think Pair Share

Attributes Student Behaviors

Strata 1. Fully engaged 2. Mostly engaged 3. Sometimes engaged 4. Never engaged

No. Study   Phase 1 Phase 2  Phase 3 

2 Tracking Student’s problem posing ability

Name Pre test Question Quality Post test

Attributes Score Rubric difficulty Score Score

Strata 1. High2. Medium3. Low

No. Study   Phase 1 Phase 2  Phase 3 

2 Tracking Student’s problem posing ability

Name Pre test Question Quality Post test

Attributes Score Rubric difficulty Score Score

Strata 1. High2. Medium3. Low

No. Study   Phase 1 Phase 2 

2 Tracking Student’s problem posing ability

Name Pre test Post Test

Attributes Score Score

Strata 1. High2. Medium3. Low

No. Study   Phase 1 Phase 2 

3 Frustration instances and motivational message in an ITS

Name Without motivational message

With motivational message

Attributes Number of frustration instances per session

Strata 1. 2 to 32. 4 to 5 3. 6

1. 0 to 12. 2 to 33. 4 to 54. >6

No. Study   Phase 1 Phase 2 

4 Tracking Mentee performance

Name Idea Planning Study Planning

Attributes Rubric score

Strata 1. High 2. Medium3. Low4. Very low

No. Study   Phase 1 Phase 2 

5 Student’s perception on LAMP framework

Name Question Posing Receiving Answer

Attributes Likert scale Perception

Strata 1. Agree 2. Neutral3. Disagree

No. Study   Phase 1 Phase 2  Phase 3 

1 Quantifying Student Engagement

Name Think Pair Share

Attributes Student Behaviors

Strata 1. Fully engaged 2. Mostly engaged 3. Sometimes engaged 4. Never engaged

2 Tracking Student’s problem posing ability

Name Pre test Question Quality Post test

Attributes Score Rubric difficulty Score

Score

Strata 1. High2. Medium3. Low

3 Frustration instances and motivational message in an ITS

Name Without motivational message

With motivational message

Attributes Number of frustration instances per session

Strata 1. 2 to 32. 4 to 5 3. 6

1. 0 to 12. 2 to 33. 4 to 54. >6

4 Tracking Mentee performance

Name Idea Planning Study Planning

Attributes Rubric score

Strata 1. High 2. Medium3. Low4. Very low

5 Student’s perception on LAMP framework

Name Question Posing Receiving Answer

Attributes Likert scale Perception

Strata 1. Agree 2. Neutral3. Disagree

Application

Exploratory Analysis

Presented a student’s engagement model.

Confirmatory Analysis

Confirmation of the qualitative data collected by zooming in the statistical distribution

Represent trends

Decrease in frustration levels

Represent trends

Increase in mentee performance

Exploratory Analysis

The perception trend is checked to further investigate

19 slides of discussion

Issues

While presenting how should it highlight the trend which the researcher wants to report?

PART 2

REVIEW PART 1

demographics

statistics

description and example

specific result

lit review

0 50 100 150 200 250 300 350 400 450 500

51

445

122

132

20

Distribution of Tables

Which of this is more effective?

demographics; 51.00

statistics; 445.00

description and example;

122.00

specific result; 132.00

lit review; 20.00

demographics

statisti

cs

description and exa

mple

specific r

esult

lit revie

w0

50100150200250300350400450500

51

445

122 132

20

How to plot this data-type?

PART 2

REVIEW PART 1&

My Doctoral research path

Data Visualization Designer Visual Analytics Trainer

2. How can visual analytics be integrated with existing analysis workflow for educational researchers?

A. What are the advantages of such a modified workflow of analysis?i. Does it provide new insights?ii. Does it make the analysis and interpretation ‘easier’?iii. Does it assist any other cognitive operation preceding a decision making task?

1. In a particular research designs apart from indicating significance of statistical difference what more relevant information can we explicate from the collected data?

A. What is the nature of effective representations for conveying educational research datasets?i. What is the kind of questions asked on Educational Datasets?ii. What is the current trend in reporting evidence to support the research?

Data Visualization DesignerResearch Questions

1. During the training session of data visualization process to an educational researcher:A. what are the modules that are in scope?B. what instructional strategies is effective for a contact workshop?

Visual Analytics Trainer

2. What are the effects of affordances that the visualization tool provides on developing skillset of that tool?

A. Is there scope of development of alternate conception about topics of effective visualization because of the affordances in the tool?

3. What cognitive model can help understand the operations during data visualization and interpreting visual representation?

Research Questions

JAN feb mar apr may jun julyaug sep oct nov dec

JAN

• Refine SAT diagram.• Identifying theoretical framework to define effectiveness and efficiency for SAT diagram

to explicate insights in data.• Investigate applicability to other ET research designs.• Check Visualization course and find concepts relevant to apply and

find solution to educational research problems.

• Developing application for generating SAT diagram• Developing teaching learning strategies to develop skillset for using visualizing tools.• Conducting a visual data representation workshop for educational dataset.

Data Representation?

“Here is my secret. It is very simple. It is only with the heart that one can see rightly

What is essential is invisible to the eye.”

- Antoine de Saint Exupéry

Thank you