engineering student problem solving processes and text-to-diagram translations

1

Engineering student problem solving

processes and text-to-diagram translations

Nikita Dawe (996562116)

Supervisor: Professor Susan McCahan

MIE498 Full-year Thesis: Engineering Education Department of Mechanical and Industrial Engineering

University of Toronto April 2014

2

i. Abstract This engineering education thesis investigates problem solving processes in the context of student

learning styles, with a focus on text-to-diagram translations. An exploratory study was conducted in

which participants performed a think-aloud problem solving activity; written work was analyzed for

accuracy, and verbal transcripts were coded and analyzed qualitatively. Results indicate that students have

not been taught explicit problem solving processes or representational tools, and students with very strong

visual or verbal learning preferences will perform less accurate translations. These findings have

implications for engineering educators and students and should be investigated further to inform the

development of educational interventions.

3

ii. Acknowledgements I would like to thank Professor Susan McCahan for providing me with the opportunity to pursue an

engineering education thesis on a topic that intrigued me. I am grateful for the guidance and feedback

offered throughout this project.

I would also like to thank the students who volunteered to participate in the study; their willingness to

find time for the research activity between academic and extracurricular commitments is appreciated. It is

not always easy to verbalize internal thoughts and reasoning, but the think-aloud material provided useful

data for analysis and interpretation.

Thank you to Matt Strohack in the Engineering Communication Program for reviewing my thesis when it

was very much a work-in-progress.

Finally, I am grateful to friends, peers, and mentors for their support and motivation.

4

Table of Contents i. Abstract ..........................................................................................................................................2

ii. Acknowledgements.........................................................................................................................3

List of tables ......................................................................................................................................7

1 Introduction ................................................................................................................................8

2 Literature Review ........................................................................................................................9

2.1 Engineering Education and Problem Solving .........................................................................9

2.1.1 Problem Solving ...........................................................................................................9

2.2 Problem Solving Models ....................................................................................................10

2.2.1 The text to diagram to symbol (TDS) algorithm ............................................................10

2.2.2 The Integrated Problem Solving (IPS) Model ...............................................................11

2.3 Learning Styles and Teaching Practices ...............................................................................12

2.3.1 Student Learning Styles ..............................................................................................12

2.4 Learning Style Categorization Tools ...................................................................................13

2.4.1 The Kolb Learning Style Inventory (LSI) .....................................................................14

2.4.2 The Myers-Briggs Type Indicator ................................................................................14

2.4.3 The Index of Learning Styles (ILS) ..............................................................................15

2.5 Learning Styles and Problem Solving Tasks ........................................................................15

3 Objectives.................................................................................................................................16

3.1 Research Questions ............................................................................................................16

3.1.1 Are text-to-diagram translations a major source of difficulty during problem solving?.....16

3.1.2 Do expressive blocks emerge in problem solving processes? .........................................18

3.1.3 What motivates students to perform text to diagram translations? ..................................18

3.1.4 Are internal representations bound across semiotic systems? .........................................18

3.1.5 Can problem solving processes and behaviour be connected to general behaviours

associated with different student learning style profiles? ..............................................................19

4 Methods....................................................................................................................................20

4.1 Research Activity ..............................................................................................................20

4.1.1 Methodology ..............................................................................................................20

4.1.2 Rationale....................................................................................................................21

4.2 Data Analysis Methodology................................................................................................23

4.2.1 Data Collected............................................................................................................23

4.2.2 Coding of Transcripts .................................................................................................24

5

4.2.3 Coding of Written Material .........................................................................................25

4.2.4 Quantitative Analysis ..................................................................................................25

5 Analysis and Interpretation of Results.........................................................................................26

5.1 Are text-to-diagram translations a major source of difficulty during problem solving? ............26

5.1.1 Did participants perform diagramming processes quickly and early? ..............................26

5.1.2 Did participants produce accurate diagrams? ................................................................28

5.1.3 Did participants verbalize accurate representations of the problems? ..............................30

5.1.4 Were participants unsure how to represent elements of the text in their diagrams? ..........31

5.1.5 What visual vocabulary patterns emerged in participants' diagrams? Did participants

exhibit diagramming conventions? .............................................................................................34

5.1.6 Did participants have difficulty representing elements of the text in their diagrams due to

conceptual prior knowledge issues? ............................................................................................35

5.2 What motivated participants to perform text-to-diagram translations?....................................38

5.2.1 Interpretation of results ...............................................................................................38

5.3 Are internal representations bound across semiotic systems? ................................................40

5.3.1 Interpretation of results: Problem 1 ..............................................................................40

5.3.2 Interpretation of results: Problem 2 ..............................................................................40

5.4 Did expressive blocks emerge? ...........................................................................................41

5.4.1 Problem 1 ..................................................................................................................41


5.4.3 Problem 2 ..................................................................................................................42


5.5 Summary of Results ...........................................................................................................43

5.6 Connecting Interpretations to Learning Style Preferences .....................................................44

5.6.1 Balanced Visual-Verbal Preferences ............................................................................44

5.6.2 Visual Preferences ......................................................................................................45

5.6.3 Verbal Preferences......................................................................................................45

5.6.4 Strong Visual-Verbal Preferences ................................................................................46

5.6.5 Sequential-Global Preferences .....................................................................................46

6 Conclusions ..............................................................................................................................47

6.1 Research Question Outcomes .............................................................................................47

6.1.1 What motivated participants to perform text to diagram translations? .............................47

6.1.2 Did participants perform diagramming processes early and quickly? ..............................47

6.1.3 Did participants produce accurate diagrams? ................................................................47

6

6.1.4 Did participants verbalize accurate representations of the problems? ..............................47

6.1.5 Were participants unsure of how to represent elements of the text in their diagrams? ......48

6.1.6 What visual vocabulary patterns emerged in participants' diagrams? Did participants

exhibit diagramming conventions? .............................................................................................48

6.1.7 Did participants have difficulty representing elements of the text in their diagrams due to

conceptual prior knowledge issues? ............................................................................................48

6.1.8 Are internal representations bound across semiotic systems? .........................................48

6.1.9 Did expressive blocks emerge? ....................................................................................48

6.1.10 Are text-to-diagram translations a major source of difficulty during problem solving?.....49

6.2 Revisiting the Thesis Objectives .........................................................................................49

6.3 Implications.......................................................................................................................50

6.4 Limitations and Future Work ..............................................................................................51

7 Works Cited..............................................................................................................................54

8 Bibliography .............................................................................................................................55

Appendices

Appendix A: Problem Solving Activity Problems ............................................................................ 56

Appendix B: Recruitment Email..................................................................................................... 57

Appendix C: Participant Information and Learning Style Profiles .................................................... 58

Appendix D: Terminology ............................................................................................................ 59

Appendix E: Iterative Coding Terms .............................................................................................. 60

Appendix F: TDS and IPS Coding of Transcripts ............................................................................ 61

Appendix G: Problem Solutions ..................................................................................................... 62

Appendix H: Rubrics for Analysis of Diagram Accuracy ................................................................. 64

Appendix I: Participant Worksheets ............................................................................................... 65

7

List of tables Table 1: Problem 1 timing data ..........................................................................................................26

Table 2: Problem 2 timing data ..........................................................................................................27

Table 3: Analysis of written material (P1) ..........................................................................................28

Table 4: Analysis of written material (P2) ..........................................................................................29

Table 5: Accuracy of internal representations (P1) ..............................................................................30

Table 6: Accuracy of internal representations (P2) ..............................................................................31

Table 7: Evidence of prior knowledge (P1).........................................................................................36

Table 8: Evidence of prior knowledge (P2).........................................................................................36

Table 9: Expressive blocks (P1) .........................................................................................................41

Table 10: Expressive blocks (P2) .......................................................................................................42

Table 11: Summary of results ............................................................................................................43

Table 12: Participant Information......................................................................................................58

Table 13: Learning Style Profiles .......................................................................................................58

Table 14: Problem 1 rubric ................................................................................................................64

Table 15: Problem 2 rubric ................................................................................................................64

8

1 Introduction "If engineering problem solving is moving from recognition to framing and then synthesis and relies on

the TDS [text-diagram-symbol] representations to construct meaning, how is that taught, or is it taught?"

(McCracken & Newstetter, 2001).

Early undergraduate engineering courses often focus on mathematical fundamentals but students are not

explicitly guided in the important steps of understanding a problem statement and translating elements

into useful diagrams for analysis. These problem solving deficiencies impede performance in coursework

and real-world activities.

McCracken and Newstetter (2001) developed the Text-Diagram-Symbol (TDS) algorithm defining three

phases and associated semiotic systems that are involved in ideal problem solving processes. The

Integrated Problem Solving (IPS) model describes more specific processes performed during each phase,

as well as episodes in which prior knowledge may be recalled or mapped. This thesis investigates student

problem solving and focuses on:

1. understanding implicit problem solving processes

2. investigating deficiencies in text-to-diagram translations

3. exploring any capabilities and deficiencies in terms of student learning profiles

The study is of a preliminary, exploratory nature due to the project time frame and the researcher's limited

research experience. It begins with a review of engineering education literature in Chapter 2. Project

objectives and research questions that stemmed from the literature review are presented in Chapter 3.

Chapter 4 describes the research activity and methods used to analyse data. Analysis and interpretation of

problem solving processes is presented in Chapter 5. This chapter includes the connection of findings to

student learning styles in Section 5.5. Chapter 6 includes a discussion of implications, limitations, and

future work.

9

2 Literature Review

The literature review began with the review of key articles on the subjects of engineering problem

solving. Wendy Newstetter and Thomas Litzinger were identified as two experts in the field, so the next

step was to obtain more of their related work. Searches were also performed to find works by co-authors

of Newstetter and Litzinger. After gaining a foundational understanding of the research topics, literature

from engineering education publications including the International Journal of Engineering Education

(IJEE) and American Society for Education (ASEE) was identified and reviewed. As the study

progressed, related topics including visual representation and engineering education practices were also

investigated. The Felder-Silverman Index of Learning Styles (ILS) was introduced to the researcher at the

beginning of the project. Articles validating and incorporating the ILS were reviewed and other learning

style categorization tools were studied and tested. The complete literature review follows.

2.1 Engineering Education and Problem Solving

Engineering education research is a broad field with a general goal of improving the quality of engineers

by advancing educational methods. Within the area of learning mechanisms, one competency that

researchers are investigating is engineering problem solving.

2.1.1 Problem Solving

Basic problem solving is an important engineering skill that is developed in first year university courses.

A traditional textbook problem will describe a system and any external factors, then ask students to solve

for a particular outcome or scenario. Typical problems cover statics, dynamics, material flow, current

flow, and abstractions of other real-world situations.

Engineering students demonstrate mixed capabilities in problem solving. Student approaches to solving

engineering problems are frequently based on prior knowledge or memory of examples presented in

lecture. They often begin using mathematical formulae to obtain a solution without ensuring a full

10

understanding of the scenario at hand. In comparison, experts tend to translate the given verbal

information into a visual model of the complete system before moving on to mathematical calculations

(McCracken & Newstetter, 2001). This aids in the correct formulation of calculations that take into

account all relevant information in interpretation of the problem.

Suboptimal problem solving techniques do not always prevent students from obtaining a correct solution.

For simple and familiar problems, it is possible to obtain a correct or sensible solution without employing

a visual translation. Students with strong mental imagery or visualization abilities may be able to identify

suitable calculations and formulate a solution without producing a diagram on paper. While it is

understood that there are numerous methods that can lead to a solution, the creation of a visual

representation motivates the deeper understanding of complex problems that extend beyond student

knowledge (McCracken & Newstetter, 2001). The optimal problem solving approach depends on the type

of problem and desired solution, student abilities and preferences, time constraints, and various other

factors. Researchers have defined optimal problem solving techniques in problem solving models.

2.2 Problem Solving Models

Problem solving models are developed to define effective approaches to problem solving and can also be

used as a point of comparison when attempting to identify why students do not succeed in solving a

problem. General problem solving models that focus on the higher level thought processes that lead to a

solution can be applied to simple problem solving tasks where all necessary information is provided, and

can also be carried over to more complex problems where students must make assumptions or define the

problem themselves. Two relevant problem solving models are the Text-Diagram-Symbol algorithm and

the Integrated Problem Solving Model.

2.2.1 The text to diagram to symbol (TDS) algorithm

McCracken and Newstetter (2001) break down the engineering problem solving process into three

iterative phases:

11

1. Problem recognition: understanding facts and data (textual)

2. Problem framing: simplification with assumptions and hypotheses (diagrammatic)

3. Problem synthesis: validation via mathematical modeling (symbolic)

They have formalized a common algorithm employed by novices and especially experts to progress

through these stages. The text to diagram to symbol (TDS) algorithm is a community-sanctioned practice

involving transformations of information between three different representational systems. The problem

statement (text) is translated into a sketch (diagram) which is then represented by mathematical formulae

(symbols) to generate a numerical solution. This process requires comprehension and generative skills

spanning across the three semiotic forms or languages – verbal, visual, and symbolic. Novices face

challenges in eliciting meaning from the individual languages and their interactions since they are not

comfortable with the associated grammatical and semantic conventions. Teaching interventions are

needed to guide students through the problem solving translations and improve their understanding of

text, diagrams, and symbols so they can obtain the information required to solve engineering problems. In

order to develop teaching interventions, researchers need to understand the cognitive processes involved

in each problem solving phase. The need for greater understanding of student cognitive processes was a

motivational factor in the development of this research thesis.

2.2.2 The Integrated Problem Solving (IPS) Model

The Integrated Problem Solving model (Litzinger, Van Meter, Wright & Kulikowich, 2006) drew from

the TDS model to map more detailed cognitive processes involved in problem solving. This model

defines ideal analysis skills of prior knowledge, problem solving processes, and translations between text,

diagrams, and symbols. These skills are expected to be used in combination to effectively solve

engineering problems. The purpose of the IPS model is to act as a tool to find areas of difficulty in student

problem solving; student processes can be analyzed in relation to the IPS model to identify skill

deficiencies.

12

IPS developers tested the model on a small group of four students to confirm its validity as a model of

problem solving and to identify student skill deficiencies. The students were tasked with constructing free

body diagrams based on a written description and an illustration. Analysis of their actions and spoken

thoughts showed that a lack of prior knowledge in the subject domain was the primary reason they had

trouble producing a correct free body diagram. This may be due to the nature of the task; if an image of

the scenario had not been provided along with the problem statement, it is likely that the students may

have found it more challenging to translate the verbal information into diagram form. This thesis will

expand on the IPS work to investigate student processes when problems do not prescribe the use of

diagrams.

2.3 Learning Styles and Teaching Practices

IPS developers noted that the model could be used to compare deficiencies between different groups of

students. Engineering education studies frequently categorize students by discipline, gender, and

academic level. Another mode of categorization is by learning style preferences.

2.3.1 Student Learning Styles

Educators recognize that students learn by individual methods based on their cognitive processes, inherent

strengths and weaknesses, and learning experience. These characteristics influence the way students

absorb, retain, and apply knowledge. The traditional methods of engineering course delivery, such as

presenting material in a lecture environment, are not effective for all students. Likewise, testing methods

do not always enable students to demonstrate the extent of their knowledge and ability. In the interest of

providing engineering students with the best possible educational experience and preparing them for

activities beyond the classroom, researchers are continuously investigating existing processes and

developing new methods for the delivery of undergraduate engineering education.

Engineering teaching methods also vary with instructor preferences. Some instructors are motivated to

vary their activities to engage students while others are unwilling to implement new methods and expect

13

students to adapt to their existing preferences. Instructors sometimes criticize students for not fully

applying themselves to their studies, while students feel that lectures and other course activities are not an

effective use of their time. Researchers need to intervene to help identify a satisfactory learning

environment for students and instructors.

If engineering education specialists can develop effective, well-defined teaching processes that suit

student preferences then instructors may be more willing to change the way they deliver courses. This

should increase participation rates as students will then regard the education process in a more positive

light. Instructors will benefit as student engagement will make their jobs more enjoyable, and students

will benefit as the quality and efficiency of their learning will improve. The first step towards this

improvement requires an understanding of student learning styles.

2.4 Learning Style Categorization Tools

Student learning practitioners have created and adopted various tests to categorize student learning

preferences. These include the Kolb Learning Style Inventory, Myers-Briggs Type Indicator, and

Inventory of Learning Styles (ILS) tool. These tools identify where students fall on particular continuums

in terms of their preferred methods of absorbing and understanding information, retaining information,

generating ideas, and solving problems. The categorization tools are based on psychological theory and

scientific studies.

Despite the extensive research that has been performed, there is still some debate regarding the validity of

aligning teaching styles with learning style preferences (Newcombe & Stieff, 2012). There are other

internal and external factors that influence student learning and it is challenging to prove that changes to

teaching styles significantly affect the quality of learning. Nonetheless, it is still important to consider the

audience when determining the content and style of information delivery. Similarly, it is well-accepted

that traditional assignments and testing methods are not always the best tools for the application and

14

evaluation of engineering material. By understanding how students think, engineering education

researchers can begin to identify better ways of helping students to grasp challenging technical concepts.

2.4.1 The Kolb Learning Style Inventory (LSI)

The Kolb LSI (Kolb, 1999) asks users to complete twelve sentences by ranking how well four different

phrases describe their way of learning. The resulting scores denote an individual's reliance on the four

learning modes of Concrete Experience, Reflective Observation, Abstract Conceptualism, and Active

Experimentation. Concrete Experience and Abstract Conceptualism relate to how an individual takes in

experience; Active Experimentation and Reflective Observation are different ways of handling

experience. The ideal learner would employ a balance of all four modes.

A comparison between the two types of scores places someone in one quadrant of the Kolb learning style

type grid. The quadrants distinguish between Diverging, Assimilating, Converging, and Accommodating

learners. These learning styles are useful when considering the higher level methods used to deliver

course content to students and to test their knowledge. The Kolb LSI is not so useful for the purpose of

investigating student techniques in the problem solving process.

2.4.2 The Myers-Briggs Type Indicator

The Myers-Briggs Type Indicator (MBTI) produces scores based on a multiple-choice survey. It reports

preferences on four dichotomies and categorizes people into one of sixteen types. The extraversion-

introversion (E-I) dichotomy defines where individuals prefer to focus their attention and whether they

gain energy from external or internal sources. The sensing-intuition (S-N) dichotomy describes how

individuals take in information. The thinking-feeling (T-F) dichotomy differentiates typical methods of

decision making. The judging-perceiving (J-P) dichotomy indicates how an individual is oriented to the

world around them. None of the MBTI dichotomies would suggest specific differences in areas of

problem solving performance that this project is focused on. Extraverted participants may be more

forthcoming with their thought processes in a think-aloud activity. Judging participants may be more

15

methodical in their problem solving approaches. This information was intended for use in the initial

selection of participants.

2.4.3 The Index of Learning Styles (ILS)

The ILS (Felder & Soloman) is a categorization tool created specifically for students in technical

programs. It was reworked to its current functionality by Felder and Soloman, based on the initial work of

Felder and Silverman. Expansive studies have validated the ILS as an effective model and identified

common preferences for groups of engineering students as well as students in other programs (Litzinger,

Lee, Wise & Felder, 2005). A questionnaire consisting of 44 prompts, each with two possible responses,

determines a student learning profile over four dimensions: active-reflective, sensing-intuitive, visual-

verbal, and sequential-global. Results denote a very strong, moderate, or balanced preference for one

category in each dimension. Students exhibiting a very strong preference may struggle when a learning

environment does not support that preference. Students with a mild preference will generally have a better

learning experience if the preference is supported. This thesis project compares student problem solving

across the visual-verbal and sequential-global dimensions.

2.5 Learning Styles and Problem Solving Tasks

As discussed above, problem solving requires students to integrate a range of skills to work from problem

statement to solution. Depending on a student's learning style preferences, unique capabilities may benefit

problem solving while associated deficiencies may hinder the problem solving process. In terms of the

ILS, one could expect to observe different approaches and sources of difficulty associated with sequential

versus global, and visual versus verbal students.

16

3 Objectives

The primary objective of this thesis project is to investigate the processes implicit in engineering student

problem solving, with a focus on translations from text to diagram. This objective is motivated by the

engineering education community's present understanding of problem solving methods and deficiencies,

and integrates the categorization of students by learning style preferences. Studies in these areas could

contribute to the design and delivery of undergraduate engineering courses by clarifying which problem

solving skills students need to be taught more explicitly.

3.1 Research Questions

Research questions centred about the three facets of the primary thesis objective were posed to guide the

design of the study methodology and broken down into more specific questions to focus data analysis.

3.1.1 Are text-to-diagram translations a major source of difficulty during problem

solving?

This overarching question has been investigated in previous engineering student problem solving studies.

If translations from text to diagram are a source of difficulty, students may not be successful in the

analytical problem synthesis phases of problem solving. To analyse the data in this study for evidence of

difficulty in text-to-diagram translations, the question was broken down into several sub-questions

addressing written problem solving processes as well as verbalized cognitive processes.

3.1.1.1 Do students perform diagramming processes early and quickly?

If translations from text to diagram are not a major source of difficulty during problem solving, students

should be able to perform these processes quickly before proceeding to mathematical processes.

3.1.1.2 Do students produce accurate diagrams?

For diagramming processes to be useful to the overall problem solving process, students need to produce

diagrams that are accurate representations of the problem statement. If translations from text to diagram

17

are a source of difficulty in problem solving, students may not construct accurate diagrams. In order to

analyse students' diagrammatic representation capabilities, this question can be further broken down:

Do students represent and connect elements of the problem in diagram form?

Do students omit elements of the problem in their diagrams?

Do students include irrelevant information in their diagrams?

3.1.1.3 Do students verbalize accurate representations of the problems?

Students might hold an accurate representation internally, but misrepresent the problem as they perform

translations from text to diagram if this process is a source of difficulty during problem solving.

3.1.1.4 Are students unsure of how to represent elements of the text in their diagrams?

The engineering education community recognizes that undergraduate students are not always taught

diagramming techniques or conventions (Litzinger, Van Meter, Wright & Kulkowich, 2006). This

research question seeks to investigate whether students experience uncertainty as they perform

translations from text to diagram form.

3.1.1.5 What visual vocabulary patterns emerge in student diagrams? Do students

follow diagramming conventions?

Students who have been taught or developed consistent visual vocabularies may have less difficulty

performing translations from text to diagram during problem solving. This question investigates whether

students show evidence of standard representations for common problem elements.

3.1.1.6 Do students have difficulty representing elements of textual problems in their

diagrams due to conceptual prior knowledge issues?

In the IPS model, phases of the TDS algorithm are composed of problem solving processes and the prior

knowledge necessary to perform processes (Litzinger, Van Meter, Wright & Kulkowich, 2006). Research

suggests that inadequate prior knowledge is a greater cause of student problem solving deficiencies than

poor problem solving processes. Prior knowledge includes pattern recognition, determination of a

18

problem's deeper structure, the retrieval of analogical problems, and mapping of mathematical

expressions and knowledge.

3.1.2 Do expressive blocks emerge in problem solving processes?

In their discussion of creativity in problem solving, Wankat and Oreovicz define an expressive block as

an inappropriate language path (Wankat & Oreovicz, 1993). Solving a problem without the appropriate

diagram or based on an inaccurate diagram is a form of expressive block, as the problem is not suitably

represented for translation to symbolic synthesis processes. If text-to-diagram translations are a source of

difficulty, the diagrams produced may act as expressive blocks.

3.1.3 What motivates students to perform text to diagram translations?

Participants were not explicitly instructed to create diagrams in the problem solving activity. The TDS

algorithm describes an ideal problem solving method, and the IPS model was only confirmed to represent

student problem solving behaviour based on a small study that instructed participants to create diagrams

(Litzinger, Van Meter, Wright & Kulikowich, 2006). This research question investigates whether students

use diagrams in their typical problem solving processes, or are motivated to do so by the nature of the

problem or challenges during non-diagrammatic problem solving processes.

3.1.4 Are internal representations bound across semiotic systems?

One cognitive factor that could have an influence on problem solving is the nature of internal

representations. This research question investigates whether information represented using one semiotic

system is tied to its representations in other semiotic systems, or is not inherently bound. If internal

representations are bound across semiotic systems, then a diagrammatic process might provoke symbolic

representations of the same information.

19

3.1.5 Can problem solving processes and behaviour be connected to general

behaviours associated with different student learning style profiles?

For the comparison of problem solving behaviours, students can be categorized by different demographic

factors. In this study, students are categorized by ILS learning style profiles to investigate whether

capabilities and deficiencies have any relation to learning preferences.

20

4 Methods

An exploratory study was conducted to investigate engineering student problem solving processes,

focusing on text-to-diagram translations. A small group of five student participants performed a problem

solving activity and their performance was analyzed for emerging patterns based on written material and

audio recordings collected during the activity. The study's research questions structured the analysis

methodology.

4.1 Research Activity

The exploratory study involved a think-aloud problem solving activity in which participants verbalized

cognitive processes as they solved two physics problems using pencil and paper.

4.1.1 Methodology

After participant recruitment and selection, participants were asked to complete the ILS survey and report

results to the researcher. The problem solving activities were performed in small meeting rooms on the

University of Toronto campus. The researcher met with participants individually. Five fourth-year

engineering students completed the study. These students were peers of the researcher, but this

relationship should not have affected their performances.

Two problems from Serway and Jewett's physics textbook (2008) were used for the problem solving

activity. The first problem (P1) was a speed and acceleration scenario while the second (P2) involved

basic trigonometry. The original diagram that accompanied P2 was not provided to participants. Both

problems required minimal expert knowledge. The problems are provided in Appendix A: Problem

Solving Activity Problems. Participants were provided with an 8.5 x 11 in worksheet containing the two

problem statements, and a pencil. They were not provided with calculators, formulae, or other reference

material. They were instructed to solve the problems as they would in an exam, and to verbalize the

internal thoughts they had as they solved the problems. The researcher performed an example activity to

ensure participants understood the think-aloud technique.

21

A Dell laptop microphone was used to record verbal protocols with Audacity 1.3 software.

4.1.2 Rationale

The experiment closely followed the methods that were used to analyze student performance upon the

development of the IPS model (Litzinger, Van Meter, Wright & Kulikowich, 2006). Decisions were made

to optimize the quality of information gained by the research, given the time and resources available.

4.1.2.1 Ethics Review

The study was proposed to and accepted by an ethics review committee before it was conducted.

4.1.2.2 Recruitment

Students previously enrolled in APS443: Leadership and Leading in Groups and Organizations were

asked to volunteer to participate in the study because they completed a MBTI assessment in October

2013. A recruitment email was distributed (see Appendix B: Recruitment Email). Participants were peers

of the student author.

4.1.2.3 Choice of Participants

Participants were University of Toronto Faculty of Applied Science and Engineering students in their

fourth years of undergraduate study. Students enrolled in a leadership course were asked to volunteer to

participate in the study because they completed a MBTI assessment in October 2013. MBTI profiles were

to be used to select participants with different learning styles if there was a large pool of volunteers. Six

volunteers responded and five were able to complete the study, so MBTI profiles were not used for

selection.

The review of literature provides evidence that students struggle with problem solving at numerous points

in the process, including the text to diagram translation necessary to represent the problem scenario

visually. By selecting upper year students for the study, the researcher hoped to identify deficiencies that

have not been addressed after three or four years of problem solving experience at the university level.

22

Upper year students who demonstrate competent translation and representation skills were expected to

provide insight that could contribute to the development of problem solving teaching interventions.

4.1.2.4 Choice of Learning Style Categorization Tool

The ILS was selected for its visual-verbal and sequential-global dimensions. The questionnaire is freely

available online and validated for students in technical programs. Participants completed the ILS in their

own time and reported their results to the researcher.

4.1.2.5 Choice of Problems to Solve

Problem 1 was chosen on the assumption that some students may attempt to solve it without the use of

diagrams. The researcher intended to investigate emerging patterns in the accuracy of solutions reached

with or without diagrams.

In the textbook, there is an image of the fountain presented with Problem 2. The image was not provided

to participants so that they would be more likely to construct their own diagrams. Even if participants

were unable to recall specific trigonometric equations, their general approach should have provided useful

information about the text-to-diagram translation.

Both problems require minimal expert knowledge. They demand the application of basic mathematical

concepts that students apply to more complex problems in their upper year engineering courses. It was

anticipated that due to these choices, the problem solving activity would provide information primarily

about student processes and text-diagram-symbol translations rather than specific technical knowledge.

4.1.2.6 Think-aloud Protocol

Think-aloud protocols are enlisted in psychology studies and user testing activities to obtain inf ormation

that cannot be observed from participant actions or measurable variables. Participants are encouraged to

express their immediate thoughts to overcome this challenge. A number of engineering education studies

related to problem solving have used think-aloud protocols to understand student challenges in the same

way this research attempts to do (Litzinger, Van Meter, Wright & Kulikowich, 2006).

23

4.2 Data Analysis Methodology

Due to the small-scale, preliminary nature of the study, it was not possible to perform thorough

quantitative analysis or implement rigorous qualitative data analysis methods. Detailed research

questions were developed relating to the study objectives of understanding cognitive aspects of student

problem solving processes and analysing text-to-diagram translations. These questions were motivated by

the literature reviewed before development of the research methodology, particularly the Text to Diagram

to Symbol (TDS) algorithm and Integrated Problem Solving (IPS) model. Observation of participants'

actions during the data gathering activities confirmed that students incorporated diagrams in their problem

solving procedures and included components of the TDS and ILS models. Since the data appeared to

align with the existing frameworks, the research questions were deemed applicable for exploratory

analysis.

4.2.1 Data Collected

For each participant, the following data was collected in the course of the learning styles and problem

solving investigation:

Preliminary and demographic information (see Appendix C: Participant Information and

Learning Style Profiles):

o Age, gender, engineering discipline

o Myers-Briggs Type Indicator type as determined in APS443 course activity

Index of Learning Styles profile: ILS preference scale scores (see Appendix C: Participant

Information and Learning Style Profiles):

o Active-reflective score

o Sensing-intuitive score

o Visual-verbal score

o Sequential-global score

Problem solving activity material

24

o Audio recordings of verbal protocols

o Participant's written solutions and work (see Appendix I: Participant Worksheets)

o Observational notes

4.2.2 Coding of Transcripts

Audio recordings of participant commentaries ranged in length from 6 minutes to 25 minutes. Audio

recordings were transcribed into digital text on the same day a participant completed the problem solving

activity. After all transcripts were available, they were broken down into "segment utterances" of single

complete thoughts. Pauses, transition words, and shifts in thoughts were used to identify separations

(Litzinger, Van Meter, Wright, & Kulikowich, 2006). Time-stamps and observational notes were

incorporated at this stage, including details of the diagrams sketched and symbolic information written in

association with each segment utterance.

The single thoughts and actions were first coded by a free, iterative process (Carberry, McKenna &

Dalrymple, 2012). The researcher began identifying applicable coding terms during the problem solving

activities and audio transcription. Transcripts were coded and additional terms were added in the process.

This coding was then revised to combine terms that were too narrow and replace terms that were too

broad. The selection of iterative coding terms is listed in Appendix E: Iterative Coding Terms.

The majority of research questions related to the three semiotic systems (textual, diagrammatic, symbolic)

or the three phases (problem representation, problem framing, problem synthesis) of the TDS and IPS

models. To investigate the research questions and explore characteristics of the data, a thematic analysis

of transcript materials was implemented. Transcripts were inspected and segment utterances were

grouped based on indicators of TDS and IPS concepts. The following four separate coding systems were

used for grouping:

1. Primary semiotic system in use; textual, diagrammatic, or symbolic

2. Episodes of translations between semiotic systems

25

3. Phases of problem solving; problem representation, problem framing, or problem synthesis

4. Episodes of the IPS model problem solving components

Details of the coding terms are listed in Appendix F: TDS and IPS Coding of Transcripts.

4.2.3 Coding of Written Material

Written material was treated as an independent artefact for coding, so corresponding transcript content

was not considered. A rubric was created to analyze the accuracy of diagrams by identifying elements

from the problem statements that could be represented in diagram form. This rubric was informed by

detailed solutions to the problems as provided by the source textbook (Serway & Jewett, 2008). These

solutions are presented in Appendix G: Problem Solutions. In addition to the correct information that

could be represented or omitted, the researcher performed an informal risk analysis to identify errors and

irrelevant information students might represent. Numerical values such as velocity and height were

considered represented if they were written in close proximity to diagrams as participants constructed the

diagrams. The complete rubric is provided in Appendix H: Rubrics for Analysis of Diagram Accuracy.

The correct symbolic solutions were also obtained from the textbook. This analysis method was selected

to increase the credibility of analysis by incorporating the impartial perspective of an external source.

4.2.4 Quantitative Analysis

Data was analysed quantitatively when possible. It was not feasible to perform frequency counts of

different types of problem solving processes as participants did not translate consistent quantities of

problem information in one action or in association with one verbal protocol statement. The accuracy of

the information comprehended, translated, or represented also varied. To enable preliminary quantitative

analysis, the timing of participant processes was analysed, and written material was analysed using a

rubric as described above.

26

5 Analysis and Interpretation of Results

This section reports and interprets patterns that emerged in the analysis of student problem solving

processes.

5.1 Are text-to-diagram translations a major source of difficulty during

problem solving?

This research question was investigated through the following questions and analysis.

5.1.1 Did participants perform diagramming processes quickly and early?

To determine whether participants performed diagramming processes early and quickly in their problem

solving processes, transcripts were reviewed to identify the beginning and completion of diagramming,

discounting minor additions some participants made later in the process. The timing of these events was

compared to the overall timeframes of participants' problem solving activities. Transcript material that

preceded the main diagramming processes was inspected to identify trends in participants' actions that

occurred before diagramming.

5.1.1.1 Problem 1

The analysis of timing for Problem 1 is presented in Table 1: Problem 1 timing data.

Table 1: Problem 1 timing data

P1 S01 S02 S03 S04 S05 Average

Pre-diagramming time (s) 15.00 50.00 80.00 70.00 165.00 76.00

Diagramming time (s) 20.00 110.00 40.00 60.00 330.00 112.00

Total time (s) 135.00 330.00 480.00 720.00 1365.00 606.00

% Pre-diagramming 11% 15% 17% 10% 12% 13%

% Diagramming 15% 33% 8% 8% 24% 18%

5.1.1.2 Problem 2

The analysis of timing for Problem 2 is presented in Table 2: Problem 2 timing data.

27

Table 2: Problem 2 timing data

P2 S01 S02 S03 S04 S05 Average

Pre-diagramming time

(s) 30.00 5.00 15.00 30.00 10.00 18.00

Diagramming time (s) 15.00 145.00 75.00 65.00 290.00 118.00

Total time (s) 145.00 435.00 720.00 225.00 345.00 374.00

% Pre-diagramming 21% 1% 2% 13% 3% 5%

% Diagramming 10% 33% 10% 29% 84% 32%

5.1.1.3 Interpretation of results

For Problem 1, the time to begin diagramming ranged from 15 seconds to almost 3 minutes. On average,

participants spent 18% of their time diagramming. For Problem 2, participants took no longer than 30

seconds to begin diagramming after they began to read the problem statement. On average, they spent

32% of their time diagramming. Participants may have spent a larger portion of time diagramming in

Problem 2 because problem synthesis involved less complicated mathematical analysis than Problem 1.

S01, S02, and S03 were consistent in the proportion of time focused on diagramming for each problem,

while S04 and S05 spent smaller proportions of their total times focused on diagramming for Problem 1

than Problem 2. These participants both struggled to understand the scenario described in Problem 2.

S01 completed diagramming at least twice as fast as the other subjects, and also provided his final

solutions in less time. S05 focused on Problem 1 diagramming for triple the time of the next slowest

participant. Her diagramming was performed in two stages of similar lengths, and accounted for a quarter

of the total time spent on the problem. S05 also spent significantly more time focused on diagramming for

Problem 2, although her overall time was faster than two other participants'.

There were no time limits, and some participants persevered through problems with prompting, so these

values are not suitable for rigorous analysis. The evidence does suggest that students tend to perform

diagramming processes early, but may not complete diagramming processes quickly. Based on transcript

content, participants showed a tendency to begin diagramming before performing thorough symbolic

analysis, although S05 tried to remember Problem 1 equations before constructing a complete diagram.

28

Some students iterated between diagramming and mathematical analysis while others moved more

sequentially from diagramming to mathematical analysis.

5.1.2 Did participants produce accurate diagrams?

The rubrics provided in Appendix H: Rubrics for Analysis of Diagram Accuracy were used to determine

the accuracy of participants' diagrams. The rubrics evaluated final, completed diagrams, so included

revisions that participants made later in their problem solving activities.

5.1.2.1 Problem 1

Analysis of the accuracy of written material for Problem 1 is presented in Table 3: Analysis of written

material (P1).

Table 3: Analysis of written material (P1)

Problem 1 S01 S02 S03 S04 S05

Represent or Omit

v(car) = 45m/s; constant Represented (Textual) (Textual) (Textual) Represented

trooper starts from v = 0 No No No No Represented

1s wait time No Represented (Textual) No Represented

a(trooper) = 3m/s2 Represented (Textual) (Textual) (Textual) Represented

t = ? No No Represented (Textual) Represented

Connect

billboard = origin (d = 0) Represented Represented Represented Represented Represented

car gets ahead of trooper No Represented Represented No Represented

d(car) = d(trooper) No Represented Represented Represented Represented

Irrelevant 2D diagram, Equations

1.5t^2 - 45t - 45 = 0 t·45 = 3t^2 3t^2 - 45t - 45 = 0

3t^2 - 45t - 45

3t^2 = 45t t^2 - 15t - 15 = 0

# Diagrams 3 1 2 1 1 2

Events A, B, C

A-B A-B, B-C A-B-C A-C A-C, A-B-C

# Elements Represented 8 3 6 7 5 8


Participants did not generally produce accurate diagrammatic representations of Problem 1. Of the eight

identified elements from the problem statement that could have been represented, all participants except

29

S05 omitted one element or more. Participants also represented some elements in text form in close

proximity to the diagram rather than using the diagrammatic semiotic system. The number of diagrams

did not determine the accuracy of a participant's overall representation of the problem; S03 represented 7

elements in one diagram, while S02 represented only 6 elements in two diagrams.

The textbook indicated that the three events described in the problem statement should be represented in

three separate diagrams. Participants produced one or two diagrams, showing evidence that students have

a tendency to attempt to represent a combination of two or three events in one diagram.

5.1.2.3 Problem 2

Analysis of the accuracy of written material for Problem 2 is presented in Table 4: Analysis of written

material (P2).

Table 4: Analysis of written material (P2)


Represent or Omit

circular pool Represented Represented Represented Represented Represented

fountain at centre Represented Represented Represented Represented Represented

C = 15m Represented Represented Represented (Textual) (Textual)

θ = 55° Represented Represented Represented Represented Represented

h = ? No Represented Represented Represented ("x")

Represented

Connect

C → r (top view) Represented Represented Represented Represented Represented

θ, r → h (side view) No Error Represented Represented Represented

Irrelevant Other θs, protractors

Person, protractor

h = (15/2π)tan55° cos55 · 2.5 = h = 15/cos55°

h = (r)tan55°; C = 2πr

x = [(tan55°)·2π]/15

h = tan55°·2; r = 2

# Diagrams 2 1 5 2 1 1

Views top, side

top top, side top, side top, side top, side

# Elements

Represented

7 5 7 7 7 7

30


Participants showed a strong tendency to represent almost all relevant information. S01 did not construct

a diagram of the side view of the fountain, and did not represent the height (h) dimension that the problem

asked participants to solve for. S02 included irrelevant angles in her diagram, and S03 included a stick

figure to represent the person described in the problem.

Overall, participants tended to produce more accurate diagrams for Problem 2 than Problem 1. Participant

S01 produced the least accurate diagrams for both Problems. The four other participants produced

diagrams that were mostly accurate, although they combined textual and diagrammatic semiotic systems

to represent problem elements.

5.1.3 Did participants verbalize accurate representations of the problems?

To determine whether participants held accurate internal representations of the problems they solved

during the research activity, problem representation components of transcripts, as identified by the

Problem Framing (F) group of the IPS phase coding methodology, were examined for emerging trends

and compared against the accuracy of participants' diagrams.

5.1.3.1 Problem 1

Analysis of the accuracy of participants' internal representations of Problem 1, as determined from verbal

and diagrammatic representations, is presented in Table 5: Accuracy of internal representations (P1).

Table 5: Accuracy of internal representations (P1)

P1 S01 S02 S03 S04 S05

1.5t^2 - 45t - 45 = 0 t·45 = 3t^2 3t^2 - 45t - 45 = 0

3t^2 - 45t - 45

3t^2 = 45t t^2 - 15t - 15 = 0

# Elements Verbal 8 6 7 8 6 8

# Elements Diagram 8 3 6 7 5 8

31

5.1.3.2 Problem 2

Analysis of the accuracy of participants' internal representations of Problem 2, as determined from verbal

and diagrammatic representations, is presented in Table 6: Accuracy of internal representations (P2).

Table 6: Accuracy of internal representations (P2)

P2 S01 S02 S03 S04 S05

h = (15/2π)tan55° cos55 · 2.5 = h = 15/cos55°

h = (r)tan55°; C = 2πr

x = [(tan55°)·2π]/15

h = tan55°·2; r = 2




In all cases where participants did not represent the complete problem in their diagrams, they provided

verbal evidence of additional elements held in internal representations. This evidence often emerged at

later stages in the problem solving activity as participants performed problem synthesis processes and

revisited the problem statement to obtain information or clarify their understanding of the problem.

This data aligns with the engineering education community's impression that translations from text to

diagram are a major source of difficulty in student problem solving; students are unable to represent their

understanding of the problem using the diagrammatic system. It also suggests that students move on to

problem framing and problem synthesis phases before fully understanding the problem, or use these

phases to develop their understanding. As their understanding becomes more complete, students do not

necessarily revise their diagrams to represent additional elements of the problem.

5.1.4 Were participants unsure how to represent elements of the text in their diagrams?

To investigate whether students may produce inaccurate or incomplete diagrams because they are not sure

how to represent elements of the problem, transcripts were examined for evidence of uncertainty or

confusion during diagramming processes. This evidence was then examined for patterns in these

expressions and in consideration of the accuracy of participants' diagrams.

32

5.1.4.1 Results

S01 did not express uncertainty or confusion during diagrammatic processes yet produced less accurate

diagrams than all other participants for both problems. S03 did not express uncertainty or confusion

during diagrammatic processes either and represented 7/8 Problem 1 elements and all seven Problem 2

elements. S04 did not appear unsure during diagramming and represented 5/8 Problem 1 elements and all

seven Problem 2 elements.

S02 expressed uncertainty or confusion on five occasions during diagramming processes. S02's Problem 1

diagram represented 6/8 problem elements and her Problem 2 diagram represented all seven problem

elements; however, she was assisted with prompts from the researcher after she showed indications of

being stuck.

5.1.4.2 Interpretation of results: Problem 1

S02 hesitated and showed indications of confusion while determining how to represent the transition

between event A and event B for Problem 1: " so … the … it's one second after what I have now … this is

embarrassing." S02 represented this element of the problem in her first of two diagrams by adding an

arrow and writing "1s" above the arrow. This element was not represented by two other participants, and

represented only in textual form by a third subject. S05 did not verbalize confusion as to how to represent

the transition, but did revise components of her second diagram to clarify its representation, as shown in

Figure 1: S05 P1 diagram 2/2. This data suggests that students were unsure how to represent a transition

in which one body changed its motion while the other continued its same pattern of movement.

Figure 1: S05 P1 diagram 2/2

33

S02 constructed a second diagram for Problem 1 after reviewing the problem statement at the end of her

problem solving process, stating "actually when I think about it that means it's not the right answer so I

would actually write that's when the distance is equal - to overtake it you need like a millisecond more."

None of the other participants considered this detail, presumably because they have adopted the

convention of simplifying problems of this sort; the textbook solution did not discuss the distinction

between overtaking and drawing level with the car.

S05 was unsure about the path of the vehicles in Problem 1, initially believing that they were travelling

along different paths on a two dimensional plane: "so when this gets here he starts trying to catch him

going this way." She retained this impression as she continued her diagramming processes which caused

her to spend significantly more time diagramming that any other participant. Eventually she chose to

assume motion was occurring in one dimension.

5.1.4.3 Interpretation of results: Problem 2

Two participants were unsure how to represent the relative locations of the person and fountain in

Problem 2. S02 did not understand the term "angle of elevation" and thought it was related to the curve of

the fountain stream. She spent time visualizing and sketching before the researcher determined that she

needed prompting. S05 was unsure whether she had represented the fountain correctly: "I don't know if

the fountain is like this - I'm confused about what he is measuring here." She was also confused by the

"angle of elevation" and did not assume that the fountain was vertical.

S02 constructed a three dimensional diagram for Problem 2, and indicated that it was more difficult to

represent information in that format, stating "but how does this angle help me get this length – that's what

I don't understand […] well actually … I don't like three-d stuff." Three other participants (S03, S04, S05)

constructed diagrams in which different views were overlaid but did not exhibit uncertainty as they added

34

representations of problem elements. For these participants, the only representation common to both

views was the radius, whereas S02 was representing relations between different angles.

This data suggests that participants were unsure how to represent Problem 2 diagrammatically because

they struggled to understand elements of the statement and elicit suitable assumptions to be represented in

problem framing. Students were sure of their representational methods for elements of the problem that

they understood correctly.

5.1.5 What visual vocabulary patterns emerged in participants' diagrams? Did

participants exhibit diagramming conventions?

Written material was examined and transcripts were reviewed to clarify the intended meaning of visual

components, in order to identify patterns in visual vocabulary.

5.1.5.1 Problem 1

The participants that constructed more than one diagram did not clearly represent how they were related.

Participants used lines and arrows to represent a number of different elements in their diagrams:

S01 used a single-line arrow (→) to represent velocity and a double-line arrow (⇒) to represent

acceleration. A vertical dashed line represented the origin.

S02 used single-line arrows (→) to represent three different types of information; the event of the

trooper setting out, the trooper's acceleration, and distances travelled. A vertical line represented the

billboard.

S03 used single-line arrows (→) to represent the car's path and the trooper's acceleration, and a

dimension line (↔) to represent the equal distance.

S04 used a horizontal line with extension lines to represent the equal distance, a vertical line to

represent the billboard, and a vertical line to represent the endpoint.

S05 used dimension and extension lines to represent distance travelled and common positions , and

single-line arrows to represent direction, velocity, and accelerations.

35

Participants used different levels of detail to represent the vehicles.

5.1.5.2 Problem 2

All participants except S04 drew a circle to represent the fountain pool. S04 drew an oval instead,

combining the anticipated top and side views in one perspective diagram.

S04 and S05 overlaid views in one diagram. Neither S02 nor S03 indicated how their diagram of the

circle was related to their triangle diagram(s).

Triangle geometry:

Of the four subjects who constructed a triangle, S02 and S05 used the conventional symbol for a right

angle in the right angle triangle.

All four included a segment curve at the vertex of the angle of elevation.

S04 included extension lines on the fountain height edge but not the radius dimension.

S05 did not use extension lines to represent relevant dimensions in this problem although she did in

Problem 1.


Participants did not show evidence of a consistent visual vocabulary as a group or within their own

diagrams. The meanings of alphanumeric symbols associated with diagrams were also inconsistent.

Participants combined the use of variable symbols (e.g. θ, x) and values (15m) within single diagrams.

The simplicity of the problems may explain why technical diagramming conventions were not generally

demonstrated.

5.1.6 Did participants have difficulty representing elements of the text in their diagrams

due to conceptual prior knowledge issues?

To investigate the extent of participants' prior knowledge in the problem subject domains, evidence of

prior knowledge identified in transcripts by the IPS coding methodology was examined for general

patterns as well as trends in relation to the accuracy of participants' diagrams and solutions.

36

5.1.6.1 Problem 1

Analysis of the evidence of prior knowledge in Problem 1 is presented in Table 7: Evidence of prior

knowledge (P1).

Table 7: Evidence of prior knowledge (P1)

P1 S01 S02 S03 S04 S05

Pattern recognition (RK1) Y Y Y N Y

Determine deep structure (RK2) Y Y Y Y Y

Prior knowledge mapped (FK1) N N Y N Y

Analogical problems retrieved (FK2) Y N N N N

Mathematical expression (SK1) Y Y Y Y Y

Prior Knowledge 5 4 3 4 3 4

1.5t^2 - 45t - 45 = 0 t·45 = 3t^2 3t^2 - 45t - 45 = 0

3t^2 - 45t - 45

3t^2 = 45t t^2 - 15t - 15 = 0



5.1.6.2 Problem 2

Analysis of the evidence of prior knowledge in Problem 2 is presented in Table 8: Evidence of prior

knowledge (P2).

Table 8: Evidence of prior knowledge (P2)

P2 S01 S02 S03 S04 S05

Pattern recognition (RK1) Y Y Y Y Y

Determine deep structure (RK2) N Y Y Y Y

Prior knowledge mapped (FK1) Y Y N N N

Analogical problems retrieved (FK2) N N N N N

Mathematical expression (SK1) Y Y Y Y Y


h = (15/2π)tan55° cos55 · 2.5 = h = 15/cos55°

h = (r)tan55°; C = 2πr

x = [(tan55°)·2π]/15

h = tan55°·2; r = 2



37

5.1.6.3 Interpretation

According to the transcript data, prior knowledge was primarily embedded in problem representation

(textual) and problem synthesis (symbolic) stages. The prior knowledge components of pattern

recognition and determination of deeper structure, that should occur before students move on to problem

representation and problem synthesis, emerged throughout participants' processes and were not restricted

to earlier stages. This suggests that students have a tendency to not recognize patterns or gain a complete

understanding of the problem from interpretation of the problem statement alone, but continue to

recognize patterns and deepen their understanding of the problem during diagrammatic framing and

symbolic synthesis processes.

Patterns of mathematical expressions emerged earlier in participants' problem solving processes. This

prior knowledge is modelled to be associated with the problem synthesis stage of an effective problem

solving process, however participants verbalized or wrote down mathematical expressions during any of

the three stages. This tendency suggests that certain characteristics of the problems provoked associated

mathematical expressions before students had determined how they would use the expressions or whether

the expressions were relevant. The emergence of mathematical expressions also suggests that additional

implicit components of prior knowledge, such as pattern recognition and the retrieval of analogical

problems, were present in participants' processes but were not expressed in their verbal protocols.

Participant data indicated a strong tendency for students not to retrieve analogical problems during

problem framing, as this knowledge was only recognized during S01's problem solving process for

Problem 1. There is evidence that misconceptions in prior knowledge can produce errors in problem

solving: "underlying errors are passed on to the mental model that is constructed to solve a problem"

(Litzinger, Van Meter, Wright & Kulikowich, 2006). S01 did not produce a correct solution to either

problem, and it might be worthwhile to investigate whether analogical problems can interfere with

problem solving processes if they cause students to make the wrong assumptions about the problem to be

solved.

38

Prior knowledge is embedded in the problem solving processes of the IPS model, so the identification of

additional implicit knowledge may display different patterns. The diagramming methods students used

were presumably based on their experience with similar problems in the past; however the practices were

so ingrained they did not identify the knowledge informing the ir diagramming processes. Furthermore,

knowledge associated with pattern recognition and determination of the deep structure of the problem

could have been mapped in diagrammatic representations, but not identified since it was embedded in the

process

5.2 What motivated participants to perform text-to-diagram translations?

To identify any factors that may have motivated participants to perform text-to-diagram translations,

transcripts were reviewed to identify the beginning of diagramming, and the preceding content was

examined for meaningful patterns. Content after the completion of diagramming was also reviewed to

identify any emerging trends in participants' tendencies to refer back to the diagrams they had

constructed.

5.2.1 Interpretation of results

S01 and S04 did not express specific motivations before initiating diagramming processes. S01 simply

stated that he would draw a diagram as his next step after reading the problem statements, indicating that

diagramming is a part of his typical problem solving procedure. S04 also proceeded to diagramming after

reading the problem statements, without indicating any specific motivations for doing so. Her phrasing in

Problem 2, "first I am going to draw a picture of the fountain," could be considered evidence that she

typically performs diagramming processes after reading the problem statement.

S02, S03, and S05 expressed a need to construct diagrams to aid the problem solving process, and

indicated that their diagramming practices are motivated by characteristics of a problem. S02 began

analysing Problem 1 before deciding to construct a diagram: "How long does it take her to overtake the

car?" "okay…so the distance…" "okay I'm gonna draw it." She appeared to move directly to the

39

symbolic semiotic system but then realized that she needed to use the diagrammatic system to clarify

elements of the problem statement. S03 and S05 demonstrated similar patterns. S02 acted differently after

she read the Problem 2 text, stating "okay I need to draw this," and beginning diagramming processes

before considering problem synthesis.

S03 also demonstrated a tendency to construct diagrams reactively rather than methodically. Near the

beginning of her Problem 2 activity she said, "I'm not even finished reading this but I'm going to go ahead

and draw this circle, because […] for some reason this seems more concrete to me." As she completed her

first diagram she appeared to realize it did not represent all elements of the problem: "he's holding the

protractor like this […] to get the angle between him and the top of the fountain" "oh, I'm going to

draw a different perspective." S05 also provided evidence that she is motivated to construct diagrams at

different stages depending on the problem, stating during Problem 2, "I find these ones I might usually

draw straight away, if it starts off with a visual thing […] whereas if it was just like cars travelling, I need

to figure out the numbers first."

S01 did not represent problems diagrammatically with as much accuracy as other participants. He referred

back to his Problem 1 diagram once after its completion, and did not show evidence of referring back to

his Problem 2 diagram. It is possible that he has adopted the practice of constructing diagrams, but does

not use them to inform problem synthesis processes, at least not in the case of this research activity. S04

also produced a less complete diagram for Problem 1, but represented all elements of Problem 2. She

referred back to her diagrams more frequently that S01.

S02, S03, and S05 produced more complete diagrams overall, and also produced more correct solutions to

the problem. All three of these participants revised their diagrams as their understanding of the problem

became more complete, and they showed a strong tendency to refer back to their diagrams during

problem synthesis. These trends suggest that S02, S03, and S05 are motivated to construct diagrams when

40

they believe they need them to solve a problem successfully, whereas S01 and S04 are motivated by the

general problem solving procedure they often follow.

5.3 Are internal representations bound across semiotic systems?

To investigate informational ties across semiotic systems, diagramming processes identified in the

research activity transcripts according to the TDS coding methodology were examined for evidence of

symbolic or mathematical language and meaningful patterns in its appearance.

5.3.1 Interpretation of results: Problem 1

Three participants (S02, S03, S04) demonstrated an association between the visual representations of

bodies in motion, and the analytical technique of equating mathematical expressions. As they constructed

diagrams, they were already dividing their attention between diagrammatical representations and

symbolic representations:

S02: drawing arrows, "I need to equate them"

S03: "x-amount of distance past the billboard"

S04: "that distance travelled will be the same so both are x"

S02 and S05 interrupted diagramming to retrieve mathematical expressions even though they had not

completely determined which unknowns they would be using equations to solve for:

S02: pauses before completing P2 diagram; "okay well acceleration equals velocity times

distance…no"

S05: "I didn't get there yet because I stopped to remember the formula. I was trying to draw the car"

5.3.2 Interpretation of results: Problem 2

Diagramming geometric representations in Problem 2 provoked symbolic representations of trigonometric

functions. S01 and S02 both used cosine instead of tangent in their final solutions for the height of the

fountain. S01 explained that he selected cosine based on the location of the radius length: "a little

rhyming thing I remember – was cos x, sine y – like the way you pronounce it is how I remember it." The

41

textual or diagrammatic representations of the problem and S01's recognition that it was a trigonometry

problem seemed to provoke a jump to symbolic calculations without the construction of a diagrammatic

side view. S02 used "h" to represent the height of the fountain. Despite the construction of multiple side

views that represented elements in the correct orientation, trigonometry equations in symbolic form

appeared to override diagrammatic and symbolic representations of "height" with representations of

"hypotenuse". S02 verbalized "therefore hypotenuse equals" while writing "h = ", and was unaware of her

error even as she presented her final solution by writing "the fountain is X m high."

5.4 Did expressive blocks emerge?

Participants' final symbolic solutions were identified and compared to the correct solutions provided by

the textbook (Serway & Jewett, 2008). The correctness of solutions was examined in relation to the

accuracy of participants' diagrams for the emergence of expressive blocks.

5.4.1 Problem 1

The accuracy of participants' diagrams and their final solutions are presented in Table 9: Expressive

blocks (P1).

Table 9: Expressive blocks (P1)


# Diagrams 3 1 2 1 1 2

Events A, B, C



1.5t^2 - 45t - 45 = 0 t·45 = 3t^2 3t^2 - 45t - 45 = 0

3t^2 - 45t - 45

3t^2 = 45t t^2 - 15t - 15 = 0


None of the study participants reached the correct solution to Problem 1. S02, S03, and S05 used an

incorrect formula to calculate distance travelled by the trooper but were otherwise correct in their

solutions; they produced more accurate diagrams, representing 6/8, 7/8, and 8/8 problem statement

elements, respectively.

42

S01 and S04 constructed less accurate diagrams, representing 3/8 and 5/8 problem statement elements,

respectively. They produced solutions that were less correct as they did not account for the 1 second

transition between events A and B and the distance the car travelled in that time. These elements were

omitted from both of their diagrams. The three other subjects who did not make this error represented the

information in their diagrams.

This data indicates that expressive blocks did emerge as barriers to the correct solution of Problem 1. S01

and S04 reached incorrect solutions after constructing incomplete diagrams. S02, S03, and S05 reached

more correct solutions after constructing more accurate diagrams. This information could also be

interpreted to mean that S01 and S04 were unable to construct accurate diagrams because they had a

poorer understanding or internal representations of the problem, and their incorrect solutions were a result

of their incomplete understanding rather than expressive blocks resulting from inaccurate diagrams.

5.4.3 Problem 2

The accuracy of participants' diagrams and their final solutions are presented in Table 10: Expressive

blocks (P2).

Table 10: Expressive blocks (P2)


# Diagrams 2 1 5 2 1 1

Views top, side

top top, side top, side top, side top, side


h = (15/2π)tan55° cos55 · 2.5 = h = 15/cos55°

h = (r)tan55°; C = 2πr

x = [(tan55°)·2π]/15

h = tan55°·2; r = 2


Excluding approximations and errors during algebraic manipulation, three participants (S03, S04, and

S05) reached the correct solution to Problem 2. They had produced accurate diagrams representing all

seven elements identified.

43

The other two participants (S01 and S02) incorrectly used cosine as the trigonometric function to

calculate the height of the fountain. S01 represented 5/7 elements of the problem and constructed only a

top view of the fountain. The incorrect solution can be connected to his omission of a diagrammatic

representation that indicates the height dimension is opposite the angle of elevation. S02 represented all

seven elements but misidentified "h" as hypotenuse. She also used the circumference measurement in

place of the radius in her second diagram (despite identifying their relation) and carried this

representation through to her solution.

This data indicates that expressive blocks may have emerged as barriers to the correct solution of Problem

2. S01's incomplete diagrammatic representation and S02's incorrect diagrammatic representation

contributed to errors during problem synthesis. If the incorrect solutions were the result of expressive

blocks, these blocks can be connected to difficulties the students had in performing text-to-diagram

translations.

5.5 Summary of Results

Table 11: Summary of results provides a summary of the results discussed above.

Table 11: Summary of results

S01 S02 S03 S04 S05

ILS Seq-Glo GLO 9 GLO 7 GLO 9 SEQ 3 GLO 3

ILS Vis-Vrb VIS 7 VRB 5 VIS 3 VRB 9 VRB 1

Problem 1

1.5t^2 - 45t - 45 = 0 t·45 = 3t^2 3t^2 - 45t - 45 = 0

3t^2 - 45t - 45

3t^2 = 45t t^2 - 15t - 15 = 0

Total time (s) 135.00 330.00 480.00 720.00 1365.00

% before begin diagramming 11% 15% 17% 10% 12%

% diagramming 15% 33% 8% 8% 24%

Pre-diagramming time (s) 15 50 80.00 70.00 165.00

Diagramming time (s) 20 110.00 40.00 60.00 330.00

Number of pauses 0 5 0 4 6

Number of assists 1 2 7 10 12



# Diagrams 3 1 2 1 1 2

44

Events A, B, C


Expressive blocks Y N N Y N

Uncertainty or confusion N Y N N Y


Problem 2

h = (15/2π)tan55° cos55 · 2.5 = h = 15/cos55°

h = (r)tan55°; C = 2πr

x = [(tan55°)·2π]/15

h = tan55°·2; r = 2

Total time (s) 145 435 720 225 345

% Pre-diagramming 21% 1% 2% 13% 3%

% Diagramming 10% 33% 10% 29% 84%

Pre-diagramming time (s) 30.00 5.00 15.00 30.00 10.00

Diagramming time (s) 15.00 145.00 75.00 65.00 290.00

Number of pauses 0 8 0 0 1

Number of assists 0 7 0 2 3



Expressive blocks Y Y N N N

Uncertainty or confusion N Y N N Y


General

Motivation Procedure Need Need Procedure Need

Bound across systems Y Y Y Y Y

5.6 Connecting Interpretations to Learning Style Preferences

Data analysis identified some potential correlations between problem solving behaviours and visual-

verbal learning style preferences, as described below. Research limitations and future steps that could be

performed to obtain more informative data are discussed in Section 6.4.

5.6.1 Balanced Visual-Verbal Preferences

S03 and S05 have balanced visual-verbal preferences. They solved both problems most correctly, and

represented problem statements most accurately in diagrammatic form as well as in verbalizations of their

internal representations. This pattern suggests that individuals with balanced visual-verbal preferences

may demonstrate greater text-to-diagram translation capabilities as they are comfortable comprehending

45

and generating information in both semiotic forms. S03 and S05 spent longer than at least two other

participants in solving each problem, and produced worksheets heavy in textual, diagrammatic, and

symbolic information.

5.6.2 Visual Preferences

S01 is the participant with the strongest visual preference. He represented 3/8 Problem 1 elements in one

diagram and 5/7 Problem 2 elements in a top view diagram. He showed little evidence of referring back to

his diagrams. If S01 does have a strong visual preference as his ILS score indicates, he may have relied

primarily on visualizations as he performed the problem solving activities. He visualized the fountain

height without constructing a diagram of the side view. S01 did not solve either problem correctly,

suggesting that his visualization techniques were ineffective, or he did not comprehend all of the

information in the text-based problem statements. However he did not verbalize uncertainty or confusion

during his processes. S01 was the only participant who did not write any complete words on his

worksheet – his written material was only in diagrammatic and symbolic form. All other participants

recorded or represented some quantity of information in textual form.

S03 has the next strongest visual preference, and S01 and S03 share a very strong global preference.

Analysis did not identify meaningful similarities between S01 and S02's problem solving behaviours. The

only notable observation is that S03 demonstrated the second lowest use of written textual information.

S03 also stated that she needed to read problem statements in her head: "if I read it out loud I won't know

what I'm reading."

5.6.3 Verbal Preferences

S04 is the participant with the strongest verbal preference. She represented 5/8 Problem 1 elements in one

diagram and all seven Problem 2 elements in one perspective diagram. She did not produce a correct

solution for Problem 1, but was correct in Problem 2 except for one error in an algebraic manipulation

step.

46

S02 has a moderate verbal preference, but analysis did not identify meaningful similarities between S02

and S04's problem solving behaviours. S02 produced an incorrect solution for Problem 2 despite

representing all elements in her diagram. She made errors during problem synthesis when the symbolic

association between "h" and "hypotenuse" interfered with her representation of "height" as "h" in her

diagrams.

5.6.4 Strong Visual-Verbal Preferences

S01 (VIS7) and S04 (VRB9) have the most polarized preferences, and demonstrated some similar

patterns in their problem solving processes. They both indicated that their diagramming processes were

motivated by standard procedures whereas the other participants constructed diagrams when they

identified the need to do so. S01 may have adopted this mechanism to overcome deficiencies in

understanding textual problems which could make it more difficult to assess when diagramming is

needed. S04 may have adopted this mechanism to overcome deficiencies in internal visualizations.

Both S01 and S04 failed to represent the same key Problem 1 element ("car gets ahead of trooper") in

their diagrams, and produced identical incorrect symbolic solutions.

5.6.5 Sequential-Global Preferences

S04's preference is balanced and oriented slightly towards sequential, which corresponds with her

procedural motivation for constructing diagrams and performing other problem solving processes. Her

written material appears to be the most structured and neat.

S01, S02, and S03 have moderate to very strong global preferences but did not show evidence of

similarities in problem solving behaviour. S05 is balanced, with a slight orientation towards global; her

total activity time was significantly longer than other subjects. Inspection of her verbal transcript suggests

she desired to understand the problem scenarios entirely, but did not have structured methods for doing

so.

47

6 Conclusions

This chapter presents outcomes from the study, a discussion of potential implications for problem solving

teaching interventions, and limitations of the study that could be addressed in future work.

6.1 Research Question Outcomes

This section summarizes findings through the structure of the research questions.

6.1.1 What motivated participants to perform text to diagram translations?

A portion of students appeared to construct diagrams as an inherent stage of their problem solving

process, while the other participants represented the text in diagram form when they determined

diagrammatic representations would aid their problem solving processes.

6.1.2 Did participants perform diagramming processes early and quickly?

Participants displayed a strong tendency to perform text-to-diagram translations early in their problem

solving processes. A typical process involved reading some or all of the problem statement, annotating or

writing down information, then diagramming. Participants showed variation in the efficiency of their

diagramming processes. They often had to return to the problem statement to obtain omitted information

or clarify their understanding of the verbal information.

6.1.3 Did participants produce accurate diagrams?

The majority of participants omitted elements in the diagrams they produced for the first problem but they

generally produced accurate diagrams to represent the second problem. Some participants supplemented

incomplete diagrams with textual representations.

6.1.4 Did participants verbalize accurate representations of the problems?

Participants exhibited a tendency to verbalize representations of the problems that were more accurate

than their diagrammatic representations.

48

6.1.5 Were participants unsure of how to represent elements of the text in their

diagrams?

Some participants indicated that they were not sure how to represent elements in their diagrams. This

tendency was most evident when participants were trying to represent the relationships between a series

of events.

6.1.6 What visual vocabulary patterns emerged in participants' diagrams? Did

participants exhibit diagramming conventions?

Participants demonstrated inconsistent and unique visual vocabularies. Some participants employed the

same visual to signify several separate elements of a diagram. Technical visual notations including

dimension and extension lines, and angle notations were produced by some participants.

6.1.7 Did participants have difficulty representing elements of the text in their diagrams

due to conceptual prior knowledge issues?

The research methodology only elicited a limited understanding of the prior knowledge participants

referred to during translations from text to diagram. The problems only required knowledge within

participants' realms of experience, and participants showed a tendency to recognize prior knowledge in

problem representation and problem synthesis phases.

6.1.8 Are internal representations bound across semiotic systems?

There was a strong tendency for participants to write or verbalize symbolic representations during

diagramming, indicating that internal representations are bound across semiotic systems. In some cases

these associations were advantageous while in others they disadvantaged students.

6.1.9 Did expressive blocks emerge?

Correlations suggested that inaccurate diagrams may have been expressive blocks against successful

problem solving. The data could did not identify or eliminate other potential sources of solution errors.

49

6.1.10 Are text-to-diagram translations a major source of difficulty during problem

solving?

Based on the research and analysis methods and the findings that emerged, it appears that translations

from text to diagram are one major source of difficulty during problem solving. Students struggle to

represent problems diagrammatically, and produce incorrect solutions based on their inaccurate diagrams

which act as expressive blocks. The engineering education community shares this belief but also

recognizes other sources of difficulty.

6.2 Revisiting the Thesis Objectives

The first objective of this thesis was to understand implicit problem solving processes performed by

engineering students. The research activity and analysis provided some insight into the cognitive

processes of a small group of students, and these results can contribute to further studies that attempt to

develop an understanding of problem solving processes in the broader engineering student body.

The second objective was to investigate deficiencies in text-to-diagram translations. Results suggest that

students are unable to produce accurate diagrams because they have not explicitly been taught translation

processes or methods of representation, and if they have, they do not know how to use diagrams to their

full advantage in mathematical analysis. Again, these findings are limited by the research and analysis

methodologies of this study, but they do identify relevant areas for future studies.

The final objective was to explore problem solving capabilities and deficiencies in terms of student

learning profiles. Interpreting the results in terms of participants' ILS profiles, it seems that students with

a very strong visual preference or very strong verbal preference will be least capable at text-to-diagram

translations. Students with a sequential preference may be more likely to adopt suggested problem solving

processes.

50

6.3 Implications

This study focused on well-defined analytical problems related to concepts fourth-year engineering

students are familiar with. The three stages of the TDS algorithm and IPS model; understanding, framing,

and synthesis, are also applicable to ill-defined problems that require students to comprehend ambiguous

or incomplete information and frame their assumptions through the text-to-diagram translation process.

These are the types of problems engineers will be solving as they move beyond the well-defined problems

of their foundational years. If they are unable to competently perform translations, inaccuracies in the

diagrammatic representations of ill-defined problems may lead students with strong analytical skills to

correctly solve the wrong problem. In addition to mathematical fundamentals, engineering students need

to be taught fundamental problem solving processes that can be extended to open-ended, real-world

problems. This research project has identified preliminary implications for any teaching interventions

being developed to address student problem solving deficiencies.

Some students with a sequential learning preference might translate from text to diagram without

understanding the problem or how diagrams will help them solve it, potentially creating inaccurate or

uninformative diagrams. As stated in the literature, "many students see diagrams as merely fulfilling a

component of the assignment rather than as an essential cognitive tool" (Waller, LeDoux & Newstetter,

2013). Findings from this project suggest that students with sequential learning profiles may express such

attitudes, and ILS studies have shown engineering students to be more sequential than other populations.

Teaching interventions are needed to help a large population of engineering students understand that

diagramming is often necessary and useful.

Interpretation of the research data identified a tendency for translations from text to diagram to be a more

prominent source of difficulty for engineering students with strong visual or verbal preferences. Students

with strong visual learning profiles require teaching interventions that enable them to become more

proficient at comprehending problem statements (problem representation). Students with strong verbal

learning profiles require interventions that will enhance their ability to represent information

51

diagrammatically (problem framing). These contrasting needs pose a challenge for educators as generic

teaching interventions may not fully address these deficiencies in students' problem solving techniques.

The most challenging translation in this study involved the representation of bodies travelling at different

rates and altering their movements in a sequence of related events. Study participants struggled to capture

the motion in static diagrams even though they held accurate internal representations. It appears that

engineering students need to be explicitly taught how to identify important events and represent these

individual stages as well as their relationships.

Study participants were inconsistent in their symbolic notations on diagrams and as a group did not

demonstrate a shared visual vocabulary. One criterion for effective engineering diagrams is "the

notational system used to reference elements in the diagram should be consistent and meaningful"

(Waller, LeDoux & Newstetter, 2013). The results of this study suggest that students must be taught why

and how they should use consistent symbolic notation as they translate problem statements to useful

diagrams. Providing students with a set of standard visual vocabulary would also be an effective means of

improving translations from text to diagram, as students not have to expend cognitive effort determining

how to represent elements.

6.4 Limitations and Future Work

This thesis project was self-directed work by an undergraduate engineering student. The researcher is a

novice when it comes to scholarly research and has limited knowledge in the fields of engineering

education and cognitive psychology. This status meant that the research was chosen to be very specific in

scope. External perspectives were not consulted to ensure consistency or correctness in the transcript

codifying process, and results and interpretations generated from qualitative analysis methods represent

the biases of the researcher.

The thesis course spanned an eight-month academic year. The time-commitment for background research

and manual coding restricted the number of participants to a small quantity. Ideally the study would

52

incorporate a large number of participants in each learning style category to allow for a more meaningful

investigation. It is also important to remember that students alter their actions in response to particular

tasks and do not always demonstrate their preferred learning style.

To extend the study and obtain more definitive results, the following recommendations are suggested for

future work:

Increase the quantity of participants and variety of problems to solve.

Include participants with different levels of experience, e.g. first-year students, fourth-year

students, graduate students, professors.

Select problems that participants have addressed more recently to eliminate the anxiety that arose

in this study due to problem unfamiliarity.

Alter the problem solving activity structure to be more like a test environment; enforce time

limits, do not prompt or assist participants, and ensure participants are motivated to persevere

with problems.

Provide support materials such as calculators or basic formulae, or instruct participants to focus

on the construction of diagrams.

Record video footage to capture specific problem solving actions.

Incorporate eye tracking technology to analyse focus of attention.

Have additional researchers participate in an iterative coding process that ensures grouping is

consistent and agreed upon.

Investigate a smaller selection of research questions in more detail.

Address research questions that were beyond the scope of this study:

o Do students perform translations iteratively?

o Do students detect and resolve errors?

o Do students demonstrate expert problem solving behaviour?

o What are the cognitive requirements of the different problem solving processes?

53

Include a survey about general problem solving processes and attitudes towards diagramming in

problem solving.

54

7 Works Cited Carberry, A. R., McKenna, A. F. & Dalrymple, O. O., "Eliciting students' interpretations of

engineering representations." 2012 ASEE Annual Conference. (2013): n. page. Web.

<http://www.asee.org/public/conferences/8/papers/3218/view>.

Felder, R. M., & Soloman, B. A. (n.d.). Index of Learning Styles (ILS). Retrieved from

http://www4.ncsu.edu/unity/lockers/users/f/felder/public/ILSpage.html

Kolb, D. A. (1999). The kolb learning style inventory. (Version 3 ed.). Experience Based Learning

Systems, Inc.

Litzinger, T. A., Lee, S. H., Wise, J. C., & Felder, R. M. (2005). "A study of the reliability and validity of

the Felder-Soloman Index of Learning Styles." Proceedings of the 2005 ASEE Annual

Conference.

Litzinger, T., Van Meter, P., Wright, M., and Kulikowich, J., "A cognitive study of modeling during

problem-solving: An Integrated Problem Solving model," ASEE Annual Conf. and Exp.,

Chicago, IL, June 2006.

McCracken, W. M., & Newstetter, W. C. (2001). "Text to diagram to symbol: Representational

transformations in problem-solving." FIE 2001 Reno 31st Annual Frontiers in Education

Conference, F2G-13.

Newcombe, N. S., & Stieff, M. (2012). "Six myths about spatial thinking." International Journal of

Science Education, 34(6), 955-971. doi: 10.1080/09500693.2011.588728

Serway, R. A., & Jewett, J. W. (2008). Physics for scientists and engineers. (7 ed., Vol. 1). Stamford, CT:

Cengage Learning.

Waller, A. A., LeDoux, J. M., & Newstetter, W. C. (2013). "What makes an effective engineering

diagram? a comparative study of novices and experts." 120th ASEE Annual Conference &

Exposition, Retrieved from http://www.asee.org/public/conferences/20/papers/6903/view

Wankat, Phillip C and Frank S Oreovicz. Teaching Engineering. Purdue University, 1993. 19 02 2014.

<https://engineering.purdue.edu/ChE/AboutUs/Publications/TeachingEng>.

http://www.asee.org/public/conferences/20/papers/6903/view

55

8 Bibliography

Anderson, E. E., & Taraban, R. (2008). "Comparison of recent engineering problem-solving models."

ICEE 2008, Retrieved from

http://www.ineer.org/Events/ICEE2008/full_papers/full_paper119.pdf

Case, J. M., & Light, G. (2011). "Emerging research methodologies in engineering education

research." Journal of Engineering Education, 100(1), 186-210. doi: 10.1002/j.2168-

9830.2011.tb00008.x

Felder, R. M. and Silverman L. K. (1988) "Learning and Teaching Styles in Engineering Education,"

Journal of Engineering Education, 78(7), 674-681.

Felder, R. M., & Soloman, B. A. (n.d.). Learning styles and strategies. Retrieved from

http://www4.ncsu.edu/unity/lockers/users/f/felder/public/ILSdir/styles.htm

Johri, A., Roth, W., & Olds, B. M. (2013). "The role of representations in engineering practices: Taking

a turn towards inscriptions." Journal of Engineering Education, 102(1), 2-19. doi:

10.1002/jee.20005

Jonassen, D., Strobel, J., & Beng Lee, C. (2006). "Everyday problem solving in engineering: Lessons for

engineering educators." Journal of Engineering Education, 95 (2), 139-151.

Kaplan, R. M., D.P. Sacuzzo, (2009, 7th ed.). Psychological testing: Principles, applications, and issues.

Belmont, CA: Wadsworth.

Lumsdaine, M. & Lumsdaine, E. (1995). "Thinking preferences of engineering students: Implications for

curriculum restructuring." Journal of Engineering Education, 84(2), 193-204.

Malgorzata, S. Z. & Waalen, J. K. (2002). "The effect of individual learning styles on student outcomes in

technology-enabled education." Global Journal of Engineering Education, 6(1), 35‑43.

McKenna, A. F., & Agogino, A.M. (2004). Supporting mechanical reasoning with a representational-rich

learning environment. Journal of Engineering Education, 93 (2), 97-104.

Taraban, R. C., & Anderson, E. (2011) "Using paper-and-pencil solutions to assess problem solving

skill." Journal of Engineering Education, 100(3), 498-519.

Wai, J., Lubinski, D., and Benbow, C. P., (2009), "Spatial ability for STEM domains: Aligning over 50

years of cumulative psychological knowledge solidifies its importance." Journal of Educational

Psychology, 101: 817-835.

56

Appendix A: Problem Solving Activity Problems

Problem 1 (Serway & Jewett, 2008):

A car traveling at a constant speed of 45.0 m/s passes a trooper on a motorcycle hidden behind a

billboard. One second after the speeding car passes the billboard, the trooper sets out from the billboard

to catch the car, accelerating at a constant rate of 3.00 m/s2. How long does it take her to overtake the

car?

Problem 2 (Serway & Jewett, 2008):

A high fountain of water is located at the centre of a circular pool. Not wishing to get his feet wet, a

student walks around the pool and measures its circumference to be 15.0 m. Next, the student stands at

the edge of the pool and uses a protractor to gauge the angle of elevation of the top of the fountain to be

55.0 degrees. How high is the fountain?

57

Appendix B: Recruitment Email

Dear classmates,

Please help me complete my engineering education thesis by volunteering as a research

participant. Participation will only take a couple of hours of your time and may help you with

future coursework. I am investigating the processes students with different learning styles use to

solve textbook-style engineering problems. I am selecting up to six participants from this course

since we have already determined our MBTI personality types. If you are willing to participate,

please read on.

Attached is the Informed Consent form for my study which contains full procedural details as

well as useful contact information. If you are selected to participate, you will be asked to

complete a short learning style survey online. In early January, I will then have you solve two

first year textbook problems in an individual interview. I will be analyzing your written work and

an audio recording of the problem solving activity.

Please read over the Informed Consent attachment carefully and return a signed copy to me in

person. Please respond to this email with the following contact and demographic information:

Name:

Email:

Telephone

Age:

Engineering discipline:

Gender:

MBTI:

Thank you very much,

Nikita Dawe

58

Appendix C: Participant Information and Learning Style Profiles

Table 12: Participant Information

Age Gender Discipline MBTI

22 M Civil Engineering INFP

21 F Chemical Engineering INTP

22 F Industrial Engineering ENTJ

22 F Materials Science and Engineering ESTJ

23 F Industrial Engineering ESFJ

Table 13: Learning Style Profiles

Subject Act-Ref Sen-Int Vis-Vrb Seq-Glo

S01 ACT 3 SEN 5 VIS 7 GLO 9

S02 ACT 1 INT 5 VRB 5 GLO 7

S03 ACT 1 INT 11 VIS 3 GLO 9

S04 ACT 3 SEN 7 VRB 9 SEQ 3

S05 ACT 7 SEN 1 VRB 1 GLO 3 1-3: Balanced

5-7: Moderate preference; may learn more easily if the preference is supported

9-11: Very strong preference; may experience difficulty if preference not supported

Engineering students are typically more sequential, sensing, and visual than non-engineering students

(Litzinger, Wise, & Felder, 2005). None of the participants demonstrate all three typical engineering

student preferences. The research did not seek to investigate a representative sample of the engineering

population.

59

Appendix D: Terminology

Semiotic systems: T (verbal/textual), D (diagrammatic), S (symbolic)

TDS algorithm: Text to Diagram to Symbol algorithm

IPS model: Integrated Problem Solving model

TDS and IPS phases: R (representation), F (framing), S (synthesis)

IPS components: P (problem solving), K (knowledge-driven)

Problems: P1 (trooper and speeding car), P2 (fountain height)

Problem 1 events: A (car passes trooper), B (trooper sets out), C (trooper overtakes car)

Subjects/Participants: S01-S05

60

Appendix E: Iterative Coding Terms

reading problem

annotating problem

understanding problem

assuming

sketching

writing symbols

deriving

guessing equation

calculating

reviewing symbols

reviewing sketch

process

presenting solution

thinking

remembering

forgotten

confusion

negativity

researcher question

researcher hint

61

Appendix F: TDS and IPS Coding of Transcripts

1. Semiotic System 3. Phase 4a. Problem Solving

Processes (P)

4b. Prior Knowledge (K)

Verbal (T) Problem Representation (R) Set subgoals (P2) Pattern recognition (K1)

holds a verbal representation, engages in data gathering

Planning (P3) Determine deep structure (K2)

Diagrammatic (D) Problem Framing (F) Execution of plans (P1) Prior knowledge mapped (K1)

generates hypotheses and makes assumptions about the problem and draws an external diagram to depict these understandings

Mapping givens (P2) Analogical problems retrieved (K2)

Mapping knowledge (P3)

Monitoring (P4)

Evaluation of final diagram (P5)

Symbolic (S) Problem Synthesis (S) Execution of plans (P1) Mathematical expression (K1)

translates the diagram into a set of mathematical equations

Mapping givens (P2)

Mapping knowledge (P3)

Monitoring (P4)

Evaluation of final solution (P5)

62

Appendix G: Problem Solutions

Problem 1 solution (Serway & Jewett, 2008):

63

Problem 2 solution (Serway & Jewett, 2008):

64

Appendix H: Rubrics for Analysis of Diagram Accuracy

Table 14: Problem 1 rubric

Problem 1

Represent or Omit

v(car) = 45m/s; constant Error: v(car) ≠ 45m/s, Δv(car) ≠ 0, v(trooper) = 45m/s

trooper starts from v = 0 Error: trooper starts from v ≠ 0

1s wait time Error: wait time ≠ 1s

a(trooper) = 3ms-2 Error: a(trooper) ≠ 3ms-2, wrong variable = 3ms-2

t = ? Error: solve for different measurement

Connect

billboard = origin (d = 0) Error: origin elsewhere

car gets ahead of trooper Error: trooper accelerates from location of car

d(car) = d(trooper) Error: distances not equal

Irrelevant Irrelevant information included in diagram(s)

Ideal Solution:

Solution 1.5t^2 - 45t - 45 = 0

# Diagrams 3

Events A, B, C

# Elements Represented 8

Table 15: Problem 2 rubric

Problem 2

Represent or Omit

circular pool Error: wrong shape

fountain at centre Error: fountain elsewhere, fountain not vertical

C = 15m Error: C ≠ 15m, wrong var = 15m

θ = 55° Error: θ ≠ 55°, wrong var = 55°

h = ? Error: solve for different measurement

Connect

C → r (top view) Error: radius not determined

θ, r → h (side view) Error: height not determined

Irrelevant Irrelevant information included in diagram(s)

Ideal Solution:

Solution h = (15/2π)tan55°

# Diagrams 2

Views top, side

# Elements Represented 7

65

Appendix I: Participant Worksheets

S01:

66

S02:

67

S03:

68

S04:

69

S05:

engineering student problem solving processes and text-to-diagram translations

Documents