just-in-time interactive online modules for applied ... · just-in-time interactive online modules...

Just-in-time Interactive Online Modules for Applied

Engineering Computing

Design & Development, and Efficacy Study

Lorena A. Barba & Adam M. Wickenheiser (Mechanical and Aerospace Engineering) with Ryan Watkins(Graduate School of Education and Human Development), The George Washington University.

We want to transform how computing is taught to engineers,increasing both their achievement in and their acceptance ofcomputing as a career-enabling skill. We will design interactiveeducational modules for teaching computing that

1. Are context-based and immediately applicable2. Use a high-level language (Python, and Matlab)3. Are delivered “just-in-time” via an online platform

When deployed at scale, students would complete the modules overthe course of their degree, as the computing skills are needed in thediscipline courses. We will work with instructors of these courses to develop relevant context for themodules, and to coordinate how and when they require completion of the modules and providestudents with guidance for time management. The modules will be supported by an open lab andwalk-in tutorials with trained learning assistants.

Our hypothesis is that this type of instruction will be more effective in educating computationallyskilled engineers, as evidenced by students’ increased proficiency with and use of computing in theirdisciplinary courses. We will collect data to assess the modules and test this hypothesis. We envisionproposing in the future a curricular transformation where the Introduction to Programming course isreplaced by this modular approach, and the Just-in-Time (JIT) modules would accrue academic credit.

Student focus groups for evaluation will draw from every year in the curriculum. In year 1, students willcomplete the modules as a remedial form of instruction running simultaneously with the traditionalprogramming course. In years 2 and 3, a study group of students will be selected for a “coursesubstitution” from the current programming course and complete the modules as an “independentstudy” option. The project will be evaluated using Technology Acceptance measures in pre- andpost-tests, by following the performance of the students in their subsequent discipline-courseassignments and projects that use computing, and by qualitative methods (think-aloud interviews).

We will curate and deliver the educational modules online, and all the materials will be publicly sharedand licensed under Creative Commons CC-BY 4.0 license. The platform itself will be open and usestandard technologies (Wordpress-based, with custom fields). All the text- and code-based materialswill be developed openly in a version-control repository.

1

1. Why we need to transform how we teach computing

Advances in computing are “a key driver of economic competitiveness,” as stated in the report of thePresident’s Council of Advisors on Science and Technology “Designing a Digital Future” (Dec. 2010).They are “crucial in achieving our major national and global priorities” in energy, education, healthcare,national security, and more. And they “accelerate the pace of discovery in nearly all other fields.”Projections of the US Bureau of Labor Statistics predict that the top-five STEM careers with the mostgrowth are all in computing (Adams, 2014). Meanwhile, nearly 50% of H1-B visa petitions approved infiscal year 2009 were for computer-related workers (US Dept. of Homeland Security, April 2010).Computing is a hot job market.

But are we developing the necessary expertise in the American people? K-12 education for the mostpart ignores computer science. At college level, the good news is that the number of computer-sciencegraduates has been increasing in the last few years, after a big 8-year slump that ended in 2009 (CRA,2013). The bad news is that 30% to 50% of students fail or withdraw from their introductory CS class(Porter et al., 2013). We could not find statistics for failure rates of students in other STEM majors, butthere is no reason to think they would be different (on the contrary, they could be higher, since thesestudents did not choose computing as their major).

There is evidence that even computer-science majors scarcely improve their computing skills after theintroductory courses. A notable Yale study published in 1983 (Soloway et al., 1983), using thenow-famous “rainfall problem,” had shocking results: only 14% of beginning students could solve theproblem (after having CS instruction), and only 36% of junior students succeeded. This study has beenrepeated many times, with equally appalling results (Guzdial, 2010). A more recent multi-institutional,multi-national study with 216 students testing on a language-independent computing problem had onlya 21% success rate (McCracken et al., 2001). Guzdial says: “It is hard to underestimate how muchstudents understand about programming in their first course.”

2. Methods and Research basis

This project falls under the category of a Design and Development research, including a pilot test ofthe intervention or solution that will be developed and a small-scale study of efficacy—this, in the senseof the “Guidelines for Education Research” document (NSF 13-126).

The proposed intervention is based on a learner-centered premise that to make learning moreeffective, students need to construct their knowledge. If not, and if the knowledge we want them toacquire contradicts in any way their mental models, students can merely memorize withoutunderstanding. In that case, students will not apply the new knowledge to solve problems in the future.

“Why is it so hard to learn to program?”

The well-known computer-science educator Mark Guzdial surveyed various studies that help explainwhy introductory programming courses have such low success rates: in natural-language task

2

descriptions, people don’t define iterations, but set operations instead, they are not explicit aboutloops; people use constraints, event-driven tasks and imperative programming, but they never talkabout objects (Guzdial, 2010). Therefore, an introductory computing course will present students withknowledge that contradicts their mental models. This explains why lecture-based presentation ofcomputing has low efficacy. Teaching computing to beginners needs to shift the emphasis on thelearner, providing relevant activities that will help students construct new algorithmic models.

Context-based computing

Guzdial also relates the Georgia-Tech experience with their media-computing course, concluding thata context-based presentation of programming increases student motivation and success (in theirexperience, it also reduced the success gender gap). Many discipline experts argue for context-basedcomputing. A recent report of the National Research Council (2011) on pedagogical aspects ofcomputational thinking says that “... the power of computational thinking is best realized inconjunction with some domain-specific content.” Applying this philosophy, Magana, Falk & Reese(2013) created a freshman course on computation and programming for materials scientists andengineers at Johns Hopkins University, designed to be learner-centered. Their evaluation of this coursefound that it was successful in increasing students’ perceptions of ability and their recognition of theimportance of computing in their fields.

A factor contributing to student frustration is that they do not immediately see the usefulness of whatthey are learning (and often, struggling with) in a first programming course. Successful initiatives toimprove introductory computer science often implement “programming in context.” For example, thechanges introduced in Harvey Mudd College to increase not only general retention but also studentdiversity included: replacing the programming language (from C to Python) and introducing appliedproblems with interesting contexts. Their success has been outstanding, on all fronts (Hafner, 2012).At the high school level, Beaver Day Country School has begun integrating coding projects into a widearray of subjects in lieu of a separate programming course. “We also recognize that coding is a mindset,so we don't want our students to memorize a certain list of commands within a certain programminglanguage. Instead, we want them to think about solving problems in innovative ways," says RobMacDonald, Beaver Day Country School’s Math Department Head (Larson, 2013).

Choice of language to teach introductory programming

The debate about programming languages is of historicproportions. But often in the debate the question of suitability

for teaching novices is not in focus. Back in 1971, Wirth(designer of the Pascal language) already noted that popularity

is not a good reason for choosing a language for teaching. The fact that a language is widely used inindustry has no relation to its suitability for helping a novice learn to think algorithmically. There aremore important criteria. Quoting Bertrand Meyer (2003): “A good teaching language should beunobtrusive, enabling students to devote their efforts to learning concepts,” not syntax.

3

One of the factors that contributes to student frustration in an introductory programming course is theuse of a compiled language, like C. The argument for using C is often the assumption that learning toprogram with a compiled language develops important skills, which may not be attained when learningwith a high-level language. The evidence, however, points to the contrary.

Mannila et al. (2006) analyzed a set of novice-written programs in Python, Java and C++. They foundthat, thanks to its simple syntax, much fewer programming errors were made in Python—an advantagewhen teaching novice programmers who get frustrated by syntax errors. There were also fewer logicerrors, and the code was more structured (in part, thanks to the indentation requirements of Python).And when students transitioned to a more complex language, there was no handicap due to havinglearned in a simpler language.

In Mannila & de Raadt (2006), the following criteria are offered for a language used in an introductoryprogramming course:

● it was designed with teaching in mind (simple syntax, natural semantics)● it can be used to apply physical analogies (provides multi-media capabilities)● it offers a general framework (serving as a basis for learning other languages later)● it promotes a new approach for teaching (augmented by principles, tools and libraries)

With the addition of other criteria involving the design environment (e.g., interactivity), support andavailability, these authors draft a table of ratings for 11 popular languages (chosen on the basis of acensus). The top languages for teaching according to this comparison are Python and Eiffel, followedclosely by Java. Since then, however, one of the items where Python lagged behind—providing aseamless development environment—has been reversed. Now, not only are there various full Pythondistributions (free for academic use), there are also full-featured development environments (like theCanopy product by Enthought) and we have the interactive and multi-media-capable IPythonNotebook.

We plan to develop the interactive modules using both Python and the domain-specific Matlab

product. Matlab is not mentioned in the literature about languages for teaching because it is not ageneral-purpose programming language—it is a domain-specific language for linear algebra. Theliterature most often is aimed at CS or related majors, not engineering, where Matlab is popular.Because many engineering faculty use Matlab, students have the perception that they will use it inindustry (we don’t have evidence supporting or refuting this at this time).

For numerical computing, Matlab and Python with the numerical libraries are very similar: they bothhave simple and clear syntax, many built-in functions, are interactive and produce good media. Thedifference between the two is that Python is free and open-source, while Matlab is commercial andproprietary. But given that faculty in engineering often prefer and use Matlab, we propose to developall the teaching materials in both Python and Matlab. Students will be free to choose for each modulewhich one they want to use.

4

Online delivery

Online learning is fraught with debate and misconceptions. The most common source of confusionoften comes when observers fail to distinguish between the use of digital media and online delivery forblended learning within a primarily classroom environment, versus distance learning with a curriculumdelivered completely online. While either form of online learning can be effective within variedcontexts, for undergraduate engineering education blended learning allows for the best of both worlds.The systematic and designed blending of learning permits instructors to provide content outside of theclassroom, while still maintaining a level of interaction in the classroom that is associated withhigh-quality undergraduate preparation.

Our proposal falls within the blended-learning category, with additional features: (i) the JIT modulesare (by definition) asynchronous, self-paced; (ii) the face-to-face interactions are with learningassistants, i.e., peers; and, (iii) the online materials are designed for interactivity (i.e., it’s not just onlinevideo!). This mix allows for the online modules to attend to three types of interactions that supportlearning; instructor-student, student-student, and student-content (Moore, 1989).

Because the face-to-face component of the JIT modules does not include direct instruction by a facultymember, some may argue that our proposal is a purely online-learning solution. Without the benefit ofa two-way debate about this contrast, we should give reasons to believe that online learning will workfor engineering computing. By embedding problem-based learning activities in each modules (withappropriate instructional assessments and strategies) the content will attend to several principles ofadult learning theory (Knowles, Holton, Swanson, 2005). Most notably, the content of modules willfocus on being relevant and applicable to undergraduate engineering students by aligning moduleactivities with engineering challenges they are being presented in courses.

While across-the-board online learning has been shown in research to have “no significant difference”from classroom learning in terms of student outcomes (Russell, 1999), our hypothesis is that within thecontext of programming for engineering students online learning modules with the characteristicsdescribed above will have superior student outcomes than the traditional non-blended offerings.

3. What we propose to do — JIT modules

For this project, we will develop 8 modules, 1 introductory and 7 associated with specific courses.We have budgeted for support in video production and instructional design for 10 modules, estimatingthat we will discard two samples after testing. The format of the modules is described below, and thetechnology that will be employed is discussed next. We will recruit 30 undergraduate students for theevaluation, and will track their performance in the engineering courses that use computing and theirattitudes to computing as a career-enabling tool. The students will cover all years of study, as seniorsand 5th-year B.S./M.S. can provide a retrospective assessment. We will also involve 4 undergraduatelearning assistants who will be trained by the PIs and the graduate students on the JIT modules and theexpected difficulties that the students might face.

5

Format of the modules and learning experience

The learning experience provided by the modules aims to develop not only programming skills butalgorithmic thinking, in context. The aim is to prepare students to use programming in later classes tosolve problems and develop confidence and computational literacy that will open doors foremployment, undergraduate research or graduate study.

Each module will focus on an applied problem that needs tobe solved with computing as a tool and is aligned with

problems presented in other engineering courses. Eachwill include:

● a short (5-min) motivation video,● richly-formatted and interactive documentation (text, equations, and figures)● sample code snippets that are executable, and● sub-goal labeled instructions (Margulieux et al., 2012).

Students will be instructed to run embedded code snippets, extend or modify the sample code andobtain a solution to an achievable problem. The expected output of practice tasks will be provided sostudents get immediate feedback. The end- and sub-goals of each module will be contextualizedaround the expository applied problem, driving the students towards a clear objective throughout.

As described below, the skills covered in each module will be identified in collaboration with the courseinstructors such that the application of each skill is clear and the link to other coursework is obvious. A“dig deeper” section (à la TED-Ed) will include references to textbooks or links to online materialwhere the students can learn more, especially aimed at the high-achieving students who might completethe modules more quickly.

Success in the modules will be a requirement for completion. In other words, we adopt amastery-learning approach: instead of assessing student learning at a set time and assigning a grade(traditionally based on partial credit), students will work on each module until they achieve mastery ofthe problem and concepts it uses. Learning for mastery is backed by extensive research evidence ashaving not only an overall positive effect on all students, but also reducing the achievement gap(Guskey, 2007). Various innovative competency-based programs are emerging in which the tenet ismastery, not grades. One example is College for America, whose presentation literature reads: “[the]primary form of assessment is completion of a task” (College for America, 2013). We believe thatprogramming skills are ideally assessed in this manner.

Technology for the modules.

Interactive computing with IPython Notebooks:The learning modules will each be a series of (5 to 10) short lessons, with text, multi-media and codesnippets. One version of each lesson will be written as an IPython Notebook (a Matlab alternative willbe made available, as well, but that platform provides less interactivity in its published output; see

6

below). In the IPython Notebook, one interweaves formatted text with equations (rendered withLaTeX code and MathJax), plots, images and video, and executable snippets of code. It is a web-basedapplication, running locally on the user’s computer via the browser, which connects each notebook to aPython kernel via the notebook server.

Notebooks can be shared online as web pages via the IPython Notebook Viewer (nbviewer). They canalso be shared by letting users download the notebook document (a text file of the source code) via apublic repository like GitHub. Users can then run the notebook locally, where the code snippetsbecome executable. Yet another way to share notebooks (available for just over a year now) is on “thecloud” using a Wakari server. Wakari is a hosted Python data-analysis environment. It facilitates learningto program in Python because students can run code on the cloud, without installing any software ontheir computer. They instead run code on the Wakari servers through the web browser. This reducesthe barriers to entry. It also provides a hosting solution where student-generated code (e.g., formativeor summative assessment of the modules) can be submitted, and an instructor or learning assistant canrun the exact code that the student submitted, on the cloud. We have discussed our proposal with staffat Continuum Analytics, the firm that created Wakari, and they have committed to support our projectin its development and testing (see letter of collaboration by Dr. Andy Terrel, Chief ComputationalScientist, Continuum Analytics).

Alternative version using MatlabWe will develop alternative versions of each lesson in each module using Matlab, the domain-specificlanguage for linear algebra. Matlab’s “publish” function allows translating marked-up Matlab code intoformatted HTML files for viewing in a web browser. Text and figure output is embedded directly inthese files alongside the code that generates them. The Matlab version of the modules will have thesame content as the IPython Notebooks; however, the code will not be executable in the browser (thisfunction does not currently exist in Matlab). Students will instead enter code snippets into their Matlabeditor to run it. We will revise the sub-goals to adjust to this limitation.

Instructional Design and Development

The design of the modules will be completed in collaboration with instructional designers from GW’sTeaching and Learning Collaborative, directed by co-investigator Ryan Watkins and supportedpart-time by a graduate research assistant. The design of the modules will include rigorous processesfor ensuring the alignment of module objectives, assessments, and instructional strategies. Our designfoundation will be the integrated results of learner and instructional analyses (Jonassen, Hannum,Tessmer, 1989), leading to detailed performance objectives for each module (Mager, 1962). Theobjectives will link to appropriate within-module assessments of student learning, as well asperformance assessments linked to engineering courses. The design will use grounded instructionalstrategies that have demonstrated efficacy for achieving desired student outcomes for each module ,rather than one instructional strategy for all modules regardless of specific objectives. E.g., a combina-tion of problem-based learning, scenarios, and direct instruction (see Hirumi, 2014) may be used acrossmodules to achieve different programming-related objectives. Throughout development, each modulewill undergo systematic formative evaluations to verify that the materials are not only technically

7

functional, but also effective in achieving student outcomes. The complete instructional design anddevelopment processes, and related documentation, will also be made publicly available for otherresearcher who may wish to alter aspects of one or more modules without reducing the efficacy of amodule’s (or the JIT series’) design.

Student support

A necessary component of competency-based learning is strong student support. The format forsupport in this project will be via online Q&A and walk-in tutoring opportunities (office hours). Forthe online Q&A, we will use an existing platform, e.g., Piazza, which provides statistics of usage andsearching tools. We will be able to analyze the forum to feedback into the process of iterativeimprovement of the modules.

In the first year of this project, the funded graduate student assisting the PIs will hold office hours, andthe teaching assistants of the relevant discipline courses will also be involved. The regular TAs will betrained in the computing modules and, as part of their normal duties, they will hold office hours insupport of their assigned courses, which will include assisting students with applying the programmingtools in assignments given in those courses. In the second and third years, two or three learningassistants will be recruited to support the students taking the modules as an independent-study option.They will be senior undergraduates, trained by the PIs and the graduate student assisting. The fundingfor learning assistants is not included in this proposal and will be sourced from the School orDepartment.

At full-scale deployment, we envisage the computing modules would be supported by two or threelearning assistants, who would be supervised and mentored by one instructor in charge. The differencewith a traditional computing class is that the instructor does not hold lectures, and that students are notsynchronized in their progress through the material. All students can advance at their own pace, but theinstructor would intervene when students fall too far behind or experience special circumstances.

Tracking student progress

The progress of participating students will be tracked, and a plan of intervention will be put in placewhen any student falls dangerously behind. An ideal platform for the delivery of the modules would alsocollect learning analytics. We do not include this aspect in the current proposal, but will track theparticipating students manually (via the interaction with learning assistants), or using the Blackboardlearning management system. We will investigate what technologies might be available to includelearning analytics in a fully-deployed solution, and we will discuss with educational-technology vendorsto determine if a sol.

Iterative improvement

In the first year, the graduate student working with Prof. Watkins will be working half-time ininstructional design and half-time in evaluation. Their role will be to examine current research literature

8

on e-learning and problem-based learning, as it applies to the JIT modules, and feed that into the designprocess. He/she will also conduct systematic learner analyses with the student focus groups to feed intothe improvement process, as well as work closely with instructional designers to formatively evaluatemodules throughout the design and development phases. In the second year, this student will work¼-time in instructional design and ¾-time in evaluation, for a new iteration of improvement taking intoaccount student progress and areas of difficulty with the JIT modules. In the third, and final, year thestudent will work exclusively on the evaluation of the project.

Additionally, the JIT modules will be developed openly, hosted on a GitHub repository. PI Barba isactive in social media and has reach within informal professional circles of computational scientists. Weare confident of having feedback from a wider network of both educators and software professionalsduring the development of the material, and we may even pick up a couple of volunteer contributorsalong the way. The open process of development on the web will contribute to the iterativeimprovement of the modules.

Example content

One course in the Mechanical Engineering curriculum where computing skills are requisite is“Analytical Mechanics II.” This course covers the dynamics of particles, rigid bodies, andinterconnected systems. The objectives of this course are for students to be able to identify the forcesand moments acting on a body and to describe its motion under these conditions. Students solve theresulting equations of motion analytically when possible and compare to computational solutions foundusing standard techniques such as Runge-Kutta methods.

A JIT module supporting this course would be designed around a problem in projectile motion.Students would solve the equations of motion without drag analytically, then with drag numerically(since an analytical solution does not exist). Student currently build ping-pong ball launchers and usemotion capture and a pressure plate to track the trajectory and time of impact. The module woulddevelop competencies in the following areas that supplement this project but could be completedasynchronously:

1. Symbolic solution of systems of ordinary differential equations (ODEs)2. Numerical solution of systems of ODEs using built-in solvers using Runge-Kutta methods3. Plotting analytical and numerical solutions side-by-side and quantitatively analyzing the

differences4. Motion capture from video recordings (tracking the center of the ball)5. Estimating velocity and acceleration from position data (or vice versa)6. Creating animations to visualize the motion of bodies or systems of bodies, tracking the motion

of specific points on a body such as the center of mass

The following courses are identified as targets for being served by the JIT modules. We will interactwith the faculty teaching (or having taught in the past) these courses to develop appropriateproblem-driven contexts. We have also identified a list of fundamental mathematical and scientificcomputing concepts to be included in the introductory modules. Below is listed the course subjects

9

with a few programming concepts to be supplemented:

● Calculus: Numerical and symbolic differentiation and integration, plotting curves, surfaces, andvector fields

● Linear Algebra: Matrix-vector manipulation, systems of equations, linear transformations● Dynamics: Solutions of ODEs, motion capture, animations● Statistics and Experimental Methods: Importing and filtering data, curve fitting and regression,

Fast Fourier Transform● Linear System Dynamics: Laplace Transforms, state-space solutions● Electromechanical Control System Design: Plotting root locus, Bode, and Nyquist plots

4. Evaluation of the Intervention

Technology Acceptance Model

We will apply the Technology Acceptance model, which was developed specifically for the study ofcomputer-technology acceptance. It proposes that acceptance of a technological innovation isdetermined by two judgements about the technology: perceived usefulness and perceived ease of use.Application of this model to study the acceptance of educational technology by both students andfaculty is well documented (Teo, 2011).

Magana et al. (2013) applied the Technology Acceptance model to evaluate a curricular innovation tointroduce computing to a materials science and engineering program, consisting of a new computingcourse, plus embedded computational modules in other courses. The focus of their evaluation was todetermine students’ acceptance of computation as a tool for their continued studies and future careers.We intend to follow their example.

During the first year, and as the modules are being developed, we will recruit volunteer students to testthe modules (giving them, say, a $50 gift card as compensation). Following Magana et al. (2013), we willuse a pre/post test with both Likert-survey and open-ended questions, intended to measure predictorsof future behavior in relation to use of computing in future studies and careers. Questions will addressperceived ability (to design an algorithm, to visualize data, etc.), utility, and intention to use in futurecoursework, projects, and in their careers (see some sample questions below). We will then obtain acomposite score in each of these three categories. These students will also be given a survey at the endof the semester asking them to evaluate how well the modules prepared them to do any otherprogramming work required in the supported courses. Some of the questions in this survey will askstudents to identify particularly difficult areas and how they would be better supported by the modules(for iterative improvement).

Example survey questions:From Magana, Falk & Reese (2013), some example questions in applying the Technology Acceptancemodel to the programming models are (answered in a Likert scale):

1. I have the ability to design an algorithm

10

2. I have the ability to write a computer program3. I have the ability to use a computer to solve a set of linear equations4. I have the ability to visualize data using a computer5. I have the ability to create a computer representation of [NAME SYSTEM]6. I have the ability to numerically solve an initial-value problem7. I have the ability to implement a numerical model based on a simple differential equation8. I feel computation will be useful in my studies9. I feel computation will be useful in my career10. I intent to seek courses that will allow me to increase my knowledge about computation11. I intend to use computation in my future career

Instructor feedback

At the end of the semester, we will engage with the instructors of the disciplinary courses to ask theirperceptions about student preparedness to apply computing for problem solving. Both the student andinstructor feedback will be used to improve later versions of the modules, to create new modules tosupport the most difficult concepts, or to provide references to additional back-up materials.

Direct evaluation

We acknowledge that the Technology Acceptance model does not directly measure the learning thathappens or the students’ ability to apply that learning, but is rather an indirect measure—by interro-gating students’ perception of ability and utility of computing, we anticipate future behavior in howstudents might use computing in their studies and careers. Students’ perception of ability is animportant measure, and in the case of programming, it is particularly useful. If students are confidentof their ability to use computing to solve problems, they are more likely to attempt it, and thus continueto get better at it.

Direct measurement of student learning outcomes is particularly problematic in the subject ofcomputing. Other subjects (notably, introductory physics) have developed over many years instrumentsfor measuring learning (e.g., the Force Concept Inventory in physics). There is no such instrument thatwe can apply in this research—at least not one that is publicly available—and developing one is aformidable undertaking that has already been subject to a full doctoral thesis (Tew, 2010). The fact ofthe matter is that, as Guzdial says, “ we currently do not have reliable and valid measures ofintroductory computing learning” (Guzdial, 2010).

The Foundational CS1 Assessment created by Tew and Guzdial (2010) is the first of its kind. It has notbeen released publicly, according to the authors, to prevent “potentially biasing participants involved inthe validation studies.” Now that the validation phase is completed, we hope that the authors will beamenable to sharing this unique instrument. We will request access to the 27-question, languageindependent test, and await her answer. If we are granted access, we will use this test to measure theperformance of students taking the traditional Introduction to Programming course, nearing the end ofthe semester (but before final exam season). For students completing the JIT modules, we will test their

11

performance in the FCS1 at different times through their progress, i.e., some students will test aftercompleting one or two foundation modules, some students will test after completing several modules,some will test after completing the whole series. If we were not granted access to the FCS1 assessment,we will evaluate the JIT modules using one of the techniques that were used in its validation:think-aloud interviews (i.e., cognitive-task analysis). This is a qualitative approach, in whichstudents are asked to verbalize their thought process as they proceed through solving a problem. Thistechnique would offer both the means to assess the students’ ability to apply module content to a uniqueproblem in the context of engineering, as well as measure the ability to use programming logic andsteps taught in the modules (versus programming techniques they may have acquired outside of themodule).

Either using the Foundational CS1 Assessment, or applying similar techniques used in its development,the evaluation of the proposed project will assess direct measures of student performance.

Experience of the investigators

PI Barba developed a series of short assignments in her Computational Fluid Dynamics (CFD) course(taught at Boston University from Spring 2010 until 2013) that lead students to program a solution tothe governing equations of fluid dynamics: the Navier-Stokes equations. The assignments were provenin the classroom to effectively take students with almost no programming experience to the pointwhere they could write a classic (but non-trivial) solution of CFD, in about four weeks of classes. InSummer 2013, Prof. Barba was invited to deliver a two-day intensive course at the “Escuela deComputación de Alto Rendimiento” (High-performance Computing School) in Mendoza, Argentina.Based on the classroom-tested assignments, she prepared a module called CFD Python (Barba, 2013a;Barba, 2013b), consisting of 12 IPython Notebooks that present the assignments incrementally usingtext and equations interspersed with code. During the two-day live training, she mentored a group of20 students of diverse backgrounds (majors of physics, engineering and computer science, all mixedtogether) through the module. Every student was challenged to prepare their own code, as they werefollowing the lessons on the IPython Notebooks, and each progressed at his/her own pace, while Prof.Barba answered questions and supported them, one-by-one. At the end of the two days, about 20% ofthe students successfully completed the full set of lessons, about 50% of the students lingered aroundstep 8 of 12, and a few straggled behind at steps 4 to 6. This experience, although anecdotal, gave Barbathe confidence to imagine a self-paced learning environment for computational engineering,supported by interactive (executable) digital notebooks.

Prof. Barba has a track record with various teaching innovations, starting with screencasts to support alecture-based course in Spring 2007, moving from whiteboard capture to digital inking, followed byvarious contributions to open courseware from 2010 onwards. She has collections on iTunes U for acomplete Fluid Mechanics course (junior), a Computational Fluid Mechanics course (senior andgraduate), and a Bio-Aerial Locomotion course (freshman). In 2011, she was the top provider ofeducational media at Boston University using the iTunes U service (as determined by the systemanalytics), and in 2012 she edited and moved the CFD content to YouTube, where her videos havecollected more than 195,000 views, collectively, since then. Also in 2012, she first applied the “flipped

12

classroom” method and since then has written and given talks widely about this pedagogical approach.She has shown a consistent engagement in educational innovation, and takes a scholarly approach thatincludes inquiry into learning theory and teaching practices, and the open dissemination of results fromher experiences and educational products.

Since 1998, Prof. Watkins has taught, written about, and conducted research on a variety of topicsassociated with instructional design and the design of online learning. He is an author of the E-learningCompanion: A student’s guide to online success, with over 150,000 copies in print and now in its 4thedition. The Companion is one of the few books that focuses on the preparation of learners for onlinecourses. In 2005 he also authored 75 E-learning Activities to help instructors find creative ways tomake online courses more interesting. In addition, he has authored or edited seven other books andmore than 90 articles on e-learning, needs assessment, or related topics. He is the developer of theWeShareScience.com online platform for sharing research. With his unique experiences as an onlineinstructor, instructional designer, and evaluator, he works to assess the ability of e-learning to meet therequirements of learning in varying contexts. He frequently consults with the World Bank one-learning development, instructional design, and program monitoring and evaluation.

Future developments

If the project evaluation supports the efficacy of the modular and context-based approach to teachingcomputing to engineers, we envisage the following possible future for this intervention. Locally, wewould soon work with our curriculum committee to make possible the permanent adoption of themodules as an integral part of the curriculum (issues like accreditation requirements will be addressedthen). Because of the discontent among the students with their current programming course, thecurriculum committee is already discussing this topic, and the situation is ripe for change. We will alsohold conversations with our colleagues in other engineering departments, share our progress andencourage them to consider similar modules, adapted to their disciplines.

More impactful is the potential of disseminating the modules to other engineering schools, so thenumber of students working on a module at a given time would increase. Then, the social-learningcomponent (Q&A platform) could move to a public, open service (say, StackExchange) and studentscould learn connectedly with others outside their institutions. With the continued support locally oflearning assistants, this will encourage and magnify the students’ learning. We would also connect withother instructors and collaborate on updates and variations of the modules, all via web-enabledcollaboration (on GitHub, etc.) There may even be scope for a new research initiative that assesses theimpact of connectivism in learning to program. Connectivism, the “learning theory for the digital age”(Siemens, 2005) may explain the success of sites for developers, such as GitHub and StackOverflow. Inshort, connectivism defines knowledge as “the set of connections formed by actions and experience”(Downes, 2007), not as something that is acquired, that can be transferred from instructor to learner. Inthis sense, it encompasses active learning, but it expands the pedagogy to a theory of learning that evenincludes knowledge stored in “non-human appliances” (Siemens, 2005)—databases, online platforms,etc. Programmers, in fact, know well how to organize knowledge that is not committed to memory, but

13

is available through social networks and online platforms. They use this every day to solve problems.

In the section “Method and Research Basis,” we start by saying that what we propose is a learner-

centered approach and we make allusion to constructivist theory of learning. We don’t wish to enterhere in the debate between constructivism and connectivism, but simply see that there are elements ofconnectivism that seem to directly apply to how programmers interact to connect their experiencethrough open online platforms. Modern computing education should recognize this culture andpossibly integrate it to the learning experience. This is a potential subject for future research.

Related online learning materials

The idea of offering short, just-in-time online modules was inspired in part by a new initiative atStanford University called JOLTs, for “Just-in-time Online Learning Tools,” launched in Summer 2013as part of the technology ventures program. The Stanford JOLTs are sparse so far, and seem to beaimed at casual support of students, rather than formal learning within the curriculum. After inspectinga few of their mathematics JOLTs, we find that they are each composed of an assortment of videos (mostare straight recordings of a short lecture at a whiteboard) with added notes in PDF and sometimes ahandful of multiple-choice questions. We will not emulate this design in our JIT modules because theirsis still a “sit and get” model of knowledge that is passed from instructor to learner.

Khan Academy has a 12-lesson “Introduction to Programming” module with Javascript that uses drawingand animation as the application context. It is a tutorial aimed at high-school students and it benefitsfrom the polished and multi-featured platform of KA. It is nice learning material, but inadequate forengineering undergraduates who will require numerical computing in their careers. KA also offers aPython-programming series of videos. These are only mini-lectures and demos on video and have nobuilt-in interactivity nor an application context.

Because of the large amount of media focus on MOOCs—with a dose of disagreement andmisunderstanding—, it’s worth explaining how our proposal differs with MOOCs. First, the JITmodules will be supported by face-to-face tutorial opportunities with learning assistants who will offeroffice hours at a physical lab. Second, while MOOCs allow consumption of the material asynchronously,the courses progress in lock-step over a 10 or 12 week period (assignments have to be turned in weekly,etc.). It is still a course, in the traditional sense. Our JIT modules will be designed for students toprogress at their own pace. Third, we are aiming at serving the credit-accruing students on campus, notthe masses (although the public may benefit from the ready and open availability of the materials online).Our project falls in the category of blended learning, with the added feature of just-in-time, masterylearning.

Broader Impacts

Unhappiness with the introduction to programming course is a common theme in engineeringcurricula. This is usually a first-year undergraduate experience, occurring before the applicationsand context are formally taught. This arrangement contributes to the frustration that can lead students

14

to transfer out of STEM majors or drop out. Our project, if successful, would serve as a model toreform the way computing is taught in other engineering departments across the nation, helping avertthe attrition of STEM students after their first year.

Computing skills are practically essential for research, and students that are more confident and skilledat programming will have more opportunities to participate in research experiences for

undergraduates. The high-achieving students that opt for graduate school will be better prepared, andmore likely to use computing effectively for their research later, pursue computational science, orcontinue improving their skills towards high-performance computing.

In sum, this project will result in computationally skilled engineers who are better prepared to enter theworkforce competitively, or if moving on to graduate studies, are ready to use computing effectively asa research tool. The JIT modules will be publicly available for others to adopt, and will developedopenly on the web. All materials will be shared under a Creative Commons license.

Results from Prior NSF funding

Lorena A. Barba

NSF award # OCI-1149784, CAREER: Scalable Algorithms for Extreme Computing onHeterogeneous Hardware, with Applications in Fluids and Biology; PI: Lorena Barba.

Total Award Amount: $550,627

Period of performance: 2/15/2012–2/14/2017

Summary: The project investigates new algebraic applications of the fast multipole method (FMM) inelliptic PDE solvers and pre conditioners, and in fast matrix-vector products within iterative methodsfor solution of algebraic equations. In incompressible fluid dynamics, Poisson solvers take the majorityof the compute time and are constrained in scalability by communication requirements. In biologicalapplications, the project looks at boundary element method solutions of bioelectrostatics and Stokesflow problems, where a dense linear system results from the discretization. We have so far developed anew application of FMM, consisting of a relaxation of the multipole truncation number as theiterations of a GMRES solver progress. This results in a 4x acceleration of the solution.

Products: Comput. Phys. Comm., 184(3):445--455 (2013); Comput. \& Fluids, 80:17--27 (2013); SIAMNews, 46(6):1 (July 2013); Comput. Phys. Comm., in press, online 4 Nov. 2013,doi:10.1016/j.cpc.2013.10.028.

Ryan Watkins — The PI has no prior NSF-funded research efforts.

Adam Wickenheiser — The PI has no prior NSF-funded research efforts.

15

References Cited

Adams, Joel C. Hot Job Market for Computer Science Graduates.Communications of the ACM,Vol. 57, No. 1, p. 19 (Jan. 2014).

Barba, L. A. (2013a) CFD Python: 12 Steps to Navier-Stokes. Blog post onhttp://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/

Barba, L. A. (2013b) CFD Python class. Bitbucket repository (IPython Notebook source code) onhttps://bitbucket.org/cfdpython/cfd-python-class/overview

College for America (2013), presentation slides in PDF (see slide 12)http://collegeforamerica.org/site_images/Reduced_College_for_America_Public_Version.4.16.13.pdf

CRA, Computing Research Association. Computing Degrees and Enrollment Trends, 2011–2012Taulbee Survey, May 2013.

Downes, Stephen (2007). What connectivism is. Post to the Connectivism Conference forum.http://halfanhour.blogspot.co.uk/2007/02/what-connectivism-is.html

Hafner, Katie (2012). Giving Women the Access Code, The New York Times, April 2, 2012http://www.nytimes.com/2012/04/03/science/giving-women-the-access-code.html

Guskey, Thomas R. (2007). Closing Achievement Gaps: Revisiting Benjamin S. Bloom's“Learning for Mastery,” Journal of Advanced Academics, vol. 19(1):8–31, Sage Publications.

Guzdial, M. (2010). Why is it so hard to learn to program? In Andy Oram and Greg Wilson,editors, Making Software: What Really Works and Why We Believe It, Chapter 7, pp. 111–124.O’Reilly Media, 2010.

Hirumi, A (2014). Grounded Instructional Strategies. Retrieve 2-3-14 fromhttp://sitios.itesm.mx/va/dide/docs_internos/docs_enc/hirumi/h01strategies.pdf

Jonassen, D. H., Hannum, W., and Tessmer, M. 1989. Handbook of Task Analysis Procedures.Westport, CT: Praeger Publishers.

Knowles, M. S., Holton, E. F., Swanson, R. A. (2005). The adult learner: The definitive classic inadult education and human resource development. Boston: Taylor & Francis Ltd.

Larson, Selena (2013). Schools Aren't Teaching Kids To Code; Here's Who Is Filling The Gap,blog post on ReadWrite, October 18, 2013.http://readwrite.com/2013/10/18/kids-learn-code-programming

1

Magana, Alejandra J., Michael L. Falk and Michael J. Reese, Jr. (2013). IntroducingDiscipline-Based Computing in Undergraduate Engineering Education. ACM Trans. Comput.Educ. 9(4), Article 39 (April 2013), 22 pages.

Magana, Alejandra J., Michael L. Falk, Mike Reese, Camilo Vieira (2013). Materials ScienceStudents’ Perceptions and Usage Intentions of Computation, 120th ASEE Annual Conference &Exposition, June 23–26, 2013. http://www.asee.org/public/conferences/20/papers/7450/view

Mager, R. (1962). Preparing instructional objectives. Atlanta: Center for Effective Performance.

Mannila, Linda and Michael de Raadt (2006). An objective comparison of languages for teachingintroductory programming. In Proceedings of the 6th Baltic Sea conference on Computingeducation research: Koli Calling 2006 (Baltic Sea '06). ACM, New York, NY, USA, 32-37.DOI=10.1145/1315803.1315811 http://doi.acm.org/10.1145/1315803.1315811

Mannila, Linda, Mia Peltomäki, Tapio Salakoski (2006). What About a Simple Language?Analyzing the Difficulties in Learning to Program. Computer Science Education 16(3): 211–228.

Margulieux, Lauren E., Mark Guzdial, and Richard Catrambone (2012). Subgoal-labeledinstructional material improves performance and transfer in learning to develop mobileapplications. In Proceedings of the ninth annual international conference on Internationalcomputing education research (ICER '12). ACM, New York, NY, USA, 71-78.http://doi.acm.org/10.1145/2361276.2361291

McCracken, M. V. Almstrum, et al. (2001). A multi-national, multi-institutional study ofassessment of programming skills of first-year CS students. SIGCSE Bulletin, Vol. 33(4), pp.125–180.

Meyer, Bertrand (1993) Towards an object-oriented curriculum. In Proceedings of the 11thInternational TOOLS Conference, pp 585–594, Prentice Hall.

Meyer, Bertrand (2003). The Outside-In method of teaching introductory programming, InPerspective of System Informatics, Proceedings of the 5th Andrei Ershov Memorial Conference,Akademgorodok, Novosibirsk, 9-12 July 2003, eds. Manfred Broy and Alexandr Zamulin, LectureNotes in Computer Science 2890, Springer-Verlag, pp. 66-78.

Moore, M.G. (1989) Three types of interaction, American Journal of Distance Education/3/2, 1-6.

National Research Council of the National Academies (2011). Report of a Workshop on thePedagogical Aspects of Computational Thinking, Committee for the Workshops onComputational Thinking. http://www.nap.edu/catalog.php?record_id=13170

NSF 13-126, National Science Foundation. Common Guidelines for Education Research andDevelopment, August 2013.

2

Porter, Leo, Mark Guzdial, Charlie McDowell and Beth Simon (2013). Success in IntroductoryProgramming: What Works? Communications of the ACM, Vol. 56, No. 8, p. 34.

President’s Council of Advisors on Science and Technology (PCAST). Designing a DigitalFuture: Federally funded research and development in networking and informationtechnology (Dec. 2010)

Russell, T. L. (1999). The no significant difference phenomenon. Chapel Hill, NC: Office ofInstructional Telecommunications, North Carolina State University.

Siemens, George (2005). Connectivism: A learning theory for the digital age, Int. Journal ofInstructional Technology & Distance Learning, Vol. 2, No. 1. http://www.itdl.org

Soloway, E., J. Bonar, et al. (1983). Cognitive strategies and looping constructs: An empiricalstudy. Communications of the ACM, Vol. 26, No. 11, pp. 853–860.

Teo, Timothy (ed.). Technology Acceptance in Education, Sense Publishers (2011)https://www.sensepublishers.com/media/1035-technology-acceptance-in-education.pdf

Tew, Allison (2010). Assessing fundamental introductory computing concept knowledge in alanguage-independent manner, PhD thesis in Computer Science, Georgia Institute ofTechnology.

Tew, Allison and Mark Guzdial (2010). Developing a validated assessment of fundamental CS1concepts. In Proceedings of SIGCSE’10, Association of Computing Machinery, pp. 97–101.

U.S. Dept. of Homeland Security, Characteristics of H-1B Specialty Occupation Workers FiscalYear 2009 Annual Report, April 2010.

Wirth, Niklaus (1971). The programming language Pascal. Acta Informatica 1: 35–63.

3

Data Management Plan

Educational materials

The educational materials (text, images and code) produced by this project in the form of JITModules will be developed openly in a public version-control repository (GitHub), and will beshared under a Creative Commons Attribution license, CC-BY 4.0.

The materials will be in two formats: IPython Notebooks (.ipynb) and Matlab program (.m) files.These files are text-based (ASCII) and thus can be version-controlled and used across platforms(Windows, Linux, Mac OS X).

Videos embedded in the modules will be published on YouTube under a Creative Commons CC0license. The original post-production videos will be in QuickTime (.mov) format and the uploadedversion will be converted to Flash (.flv) format.

The materials will be deposited in permanent online repositories, and all documents generated as aresult of this work will be either published in the peer-reviewed literature, at scholarly conferencesor deposited in permanent repositories that provide a digital object identifier (DOI) and perma-link(e.g., Figshare).

Human-subject data

This project will collect data in the form of performance tests, attitude surveys and interviews. It willnot collect demographic data nor any personally-identifying data, but it will collect data on previousbackground (pertaining to this project, e.g., programming experience), major and year of study.

According to the NSF program conditions, “IRB exemption or approval is required at the time ofthe award” (not at the time of the proposal). We will follow the George Washington Universitypolicies and compliance measures, but anticipate that an exemption is applicable, as this research:

● will be conducted under normal educational settings and will involve normal educationalpractices (research on instructional strategies and research on the effectiveness of orcomparison among instructional techniques and curricula);

● will involve educational tests, but will not collect any identifiable data and will not discloseresponses outside the research team nor would such a disclosure put any subjects at risk.

8/3/14 8:36 AMNSF FastLane :: Proposal Status

Page 1 of 2https://www.fastlane.nsf.gov/researchadmin/viewProposalStatusPane…41618&panelSummaryNumber=1&performOrg=George%20Washington%20Univ

PI/CO-PI Management -Proposal Functions | HOME

NSF Home | News | Site Map | GPG | AAG | Contact Us |FastLane Help

Change Password | Logout

Proposal Status | MAIN Organization: George Washington University

Panel Summary #1

Proposal Number: 1432340

Panel Summary: Panel Summary

Panel Summary

This project proposes to develop, pilot, and assess 5-10 interactive modules to teach programming to mechanicalengineering students. Modules will be administered Just In Time. Modules are to develop programming as well asalgorithmic skills for students. They will be used for mastery of concepts rather than a specific grade. The intent ofthe work is to eventually convert the introduction to programming class into these embedded modules throughoutthe curriculum. They also propose to focus on applied problems that need to be solved with computing as a tool andare aligned with problems in engineering courses.

Intellectual Merit

- Strengths

In general the panel felt that the proposal was innovative and the use of High level language imbedded in thecurriculum as modules would help engineers learn about programming Just-In-time in their learning process. Thecurrent method of teaching programming to engineers is not working so researching another way and providingstatistical data will help, not only mechanical engineering, but all other engineering disciplines.

- Weaknesses

One panelist did not see innovation in the project. How is this different from adding another supplementary moduleto a class? There was also confusion about how the modules would work. Will snippets of code be provided? How willstudents learn from this process?

Broader Impacts

- Strengths

The results of this project can have a great impact on all engineering fields not just mechanical engineering.

- Weaknesses

The sections were not presented in much detail. More should have been discussed on how dissemination andbroader impacts will be addressed.

Data Management Plan:

The data management plan is lacking information on securing student identifiable data. The PIs state that they willfollow GWs IRB process but nothing has been documented for the actual management of student assessment data.The PIs document how the modules will be disseminated.

Summary Statement:

This project proposes to develop, pilot, and assess 5-10 interactive modules to teach programming to mechanicalengineering students. Assessment of modules will be conducted. It is well written and plans to address a neededarea of research: how to teach engineers programming in a way to allow for better retention of programming andalgorithmic development skills. The panel felt that using context-based (solving engineering problems) and high-


Page 2 of 2https://www.fastlane.nsf.gov/researchadmin/viewProposalStatusPane…41618&panelSummaryNumber=1&performOrg=George%20Washington%20Univ

level language modules (Matlab or Python which students will be using in other classes) delivered Just-In-Time isinnovative.

This summary was read by the panel, and the panel concurred that the summary accurately reflects the paneldiscussion.

Back to Proposal Status Detail

Download Adobe Acrobat Reader for viewing PDF files

National Science Foundation

4201 Wilson Boulevard, Arlington, Virginia 22230, USATel: 703-292-5111, FIRS: 800-877-8339 | TDD: 703-292-5090

Privacy and

Security


Page 1 of 2https://www.fastlane.nsf.gov/researchadmin/viewProposalStatusRevie…0Washington%20Univ&revPropRtngCode=V&revPropRtngInd=NYNNN&revNo=1





Review #1


NSF Program: Improving Undergraduate STEM Education (IUSE)

Principal Investigator: Barba, Lorena A

Proposal Title: IUSE: IUSE Projects: Just-in-time Interactive Online Modules for Applied EngineeringComputing: Design & Development, and Efficacy Study

Rating: Very Good

REVIEW:

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

Overview This project consists of developing, piloting, and assessing interactive online modules to teach programming toengineering students.

They propose to focus on applied problems that needs to be solved with computing as a tool and is aligned withproblems in engineering courses. Each module will consists of 1. A short motivation video 2. Richly-formatted and interactive documentation 3. Sample code snippets 4. Sub-goal labeled instructions

Intellectual merit Strength The PI convincingly argues that programming course for engineers do not prepare our students. The PI proposes to teach programming to engineering students by using context-based, high-level language that aredelivered just-in-time. The hypothesis is that this type of instruction will more effective in teaching programming toengineering students.

Weakness Most likely this will teach engineering students to use programming more effectively to solve engineering problems,but the reviewer is not sure whether this method will teach the fundamental concepts more effectively thantraditional course. However, it might be worth a try, because current programming courses do not work forengineering students.

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

Strength The PI mentions that interesting programming course will help with the retention of our students, and moreopportunities to participate in research.

Weakness The PI does not propose any direct outreach activities. Because programming is an important subject, there must besome outreach activities that the PI could be involved to improve the impact of the broader impact.



Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable

Summary Statement

There is a definitely a need to improve programming courses for engineers. This may be solution in the correctdirection. Broader impact needs to be more detailed and specific.





Privacy and

Security


Page 1 of 2https://www.fastlane.nsf.gov/researchadmin/viewProposalStatusRevie…0Washington%20Univ&revPropRtngCode=G&revPropRtngInd=NNYNN&revNo=2





Review #2





Rating: Good

REVIEW:


In this proposal, 10 short lessons will be developed to teach programming. The modules will be deployed in a just-in-time format at specific points in various courses where programming skills are required to solve engineeringproblems. This is likely to be beneficial to students who are trying to do home assignments and are weak inprogramming.

Since snippets of code will be provided, it is not clear how well students will learn programming. Most of the time, ifthe code works, students do not pay attention to why it worked and are likely to make mistakes if they have to writethe code again from scratch.


The issue of broader impact or the inclusion of women and minority populations is not specifically addressed.


The proposal states that only 30 students will be impacted by this proposal, which is rather low for the amount offunding requested.

Summary Statement

In this proposal, 10 short lessons will be developed to teach programming. The approach is not transformative andbroader impact is limited.




Page 2 of 2https://www.fastlane.nsf.gov/researchadmin/viewProposalStatusRevie…0Washington%20Univ&revPropRtngCode=G&revPropRtngInd=NNYNN&revNo=2



Privacy and

Security







Review #3





Rating: Very Good

REVIEW:


This project aims to develop 5-10 modules to teach programming to mechanical engineering. Modules will beadministered Just In Time. Modules are to develop programming as well as algorithmic skills. They will be used formastery of concepts rather than a specific grade. The intent is to eventually convert the introduction toprogramming class into these embedded modules throughout the curriculum.

Strengths: The concept of teaching in modules is not new but integrating it in the curriculum to replace a real courseis. This is a very interesting project in that it will provide valuable information on whether this process could trulyhelp teach programming JIT.


The broader impacts of this project, if successful, can be huge. If the process works for Mechanical engineers at thisinstitution, it can be replicated at other institutions.

Strengths: the uniqueness of the approach is a great strength of this proposal.

Weakness: the idea is a game changer in teaching programming to non-computer scientists and faculty may beresistant to implementing it in their institutions.


Summary Statement

This project proposes to develop modules to teach mechanical engineers programming JIT throughout thecurriculum rather than through a single programming course in the first year, the idea is unique and can providevaluable data on teaching programming to mechanical engineers.







Privacy and

Security


Page 1 of 2https://www.fastlane.nsf.gov/researchadmin/viewProposalStatusRevie…0Washington%20Univ&revPropRtngCode=F&revPropRtngInd=NNNYN&revNo=4





Review #4





Rating: Fair

REVIEW:


The objective of the project is to develop and evaluate online modules to teach programming to engineeringstudents. The idea is neither new nor transformative and aims at expanding an existing approach to teachprogramming to engineering students.


The 'broader' impacts of the proposed activities are not thoroughly discussed. In fact, only a few lines are dedicatedto the broader impacts portion of the study. The team is encouraged to expand on their thoughts and to detail aplan to engage underrepresented students in the study. The budget is excessive for the proposed activities.


n/a

Summary Statement

The proposed idea is systematic and is not transformative and has been evaluated in the literature. The broaderimpacts section needs strengthening and it is very brief and does not explain how underrepresented students will beinvolved in the research.




4201 Wilson Boulevard, Arlington, Virginia 22230, USAPrivacy and


Page 2 of 2https://www.fastlane.nsf.gov/researchadmin/viewProposalStatusRevie…0Washington%20Univ&revPropRtngCode=F&revPropRtngInd=NNNYN&revNo=4

Tel: 703-292-5111, FIRS: 800-877-8339 | TDD: 703-292-5090 Security

just-in-time interactive online modules for applied ... · just-in-time interactive online modules...

Documents