assessment report viewer to meet enrollment and admissions ... · comparablysized mba programs at...

Assessment Report Viewer

Analytics (Spring/2015)

Objective 1: To meet enrollment and admissions targets

Outcomes Evidence Collected and Findings

Evaluation: Strengths andAreas for Improvement

Actions Taken to ImprovePrograms

Meet student enrollment target,which is set at 100percent ofoperating capacity for the program(currently 80 students/year)

Enrollment statisticsFINDINGS:Enrollment forAcademic Year 201415 was 86students, which exceeded ourenrollment target figure by 6students. Because we typicallyexperience some attrition, weoften admit in excess of ouroptimal capacity.

STRENGTHS: Our reputation,annual placement statistics, shorttime to degree, and our practicaland wholistic approach toprofessional education are themain reasons admitted applicantsenroll in our program over othersto which they may have beenaccepted. The high number ofapplications in the 201314 and201415 academic yearscompelled us to once againexpand the program capacity.AREAS FOR IMPROVEMENT:As analytics programs proliferate,we will need to increase ourrecruiting efforts locally andnationally, and find ways toexpand the pool of potentiallyexcellent students.

We have developed twoexperimental online courses toprepare potential applicants, whowhile otherwise qualified, arelacking adequate preparation inone or more of the subject areasnecessary to be successful in theprogram, as a way of increasingthe number of applicants to theprogram. The first was offeredsuccessfully in Spring 2015; thesecond will be offered in Fall2015.

Meet application pool target of 4:1(applicants to available seats), orbetter

Application statisticsFINDINGS:Applications reacheda record high for Academic Year201415, a 75% increase over theprior year, and three times thenumber of applications comparedto 2013 (our first cohort of 80students).

STRENGTHS: The pressattention given to the programdrives many potential applicantsto our web site. Our reputation,annual placement statistics, shorttime to degree, and our practicaland wholistic approach toprofessional education are themain reasons applicants apply toour program. AREAS FORIMPROVEMENT: Since weinvented the MS in Analytics 8years ago, many competitorshave begun programs, some quitesimilar to our own. We will needto implement and improverecruitment efforts within the stateand in targeted areas nationallyand internationally.

We are now encouraging earlyapplication, particularly fromcandidates who may needadditional academic preparation inorder to be successful in theprogram, so that we can guidethem toward prerequisites theycan complete before the start ofthe program. In this way we arehelping to grow the pool ofqualified applicants. We have alsochanged the format of MSAinformation sessions, limiting thenumber of attendees andincreasing the number of programstaff present in order to spendmore oneonone time withpotential applicants. We havedecided to replicate this approach,to the extent possible, with onlinesessions.

Meet acceptance rate target of below30percent

Enrollment statisticsFINDINGS:The program was ableto maintain its selectivity. Theacceptance rate reached an alltime low12.7%, or approximately1 in 8 applicants.

STRENGTHS: A large applicantpool ensures selectivity whichhelps to further establish the MSAprogram as a selective program ofhigh quality. AREA FORIMPROVEMENT: Ongoing effortsto increase or at the very leastmaintain the number ofapplications will be critical asmore and more programs becomeavailable.

We worked with UNCWilmingtonto give an on campuspresentation to a large audienceof interested students. We havedecided to schedule on siteinformation session/recruitingevents at our largest "feeder"institutions in North CarolinaUNC Chapel Hill, UNCWilmington, and AppalachianState University.

Meet enrollment rate target of greaterthan 80percent

Enrollment statistics FINDINGS:Of 310 applicant finalistsinterviewed for admission, 101were offered a seat in the Classof 2015. Of those offeredadmission, 86 enrolled, exceedingour enrollment rate target by fivepercent.

We have decided to provideadditional summary information toall finalist candidates, prior to theirinterviews, on the strengths of theMSA program, relative to otherprograms they may beconsidering, and an opportunity toeasily schedule time with anadmissions staff member toanswer any questions they mayhave. We have also decided topursue requiring a nonrefundabledeposit for enrolled applicants toreserve their seats in the program.

STRENGTHS: Our programenjoys an excellent reputation,and is usually a top choice foraccepted applicants. AREASFOR IMPROVEMENT: We havelost several promising applicantsto other programs after they haveaccepted enrollment into ours,universally because they wereoffered funding by the otherinstitution. Proactively providingpromising applicants additionalinformation on the relativestrengths of our programcompared to others could helpensure that students choose ourprogram over others to which theyhave applied. Requiring a nonrefundable tuition deposit toreserve their seat in the programwould dissuade many studentsfrom accepting enrollment at otherinstitutions after having acceptedenrollment to ours.

Meet student attrition target of below10percent

Enrollment statisticsFINDINGS:We had no studentattrition in Academic Year 201415.

STRENGTHS: Our admissionsprocess is one of carefulscreening. All applications arereviewed my multiple programstaff, and finalists are required tointerview either byvideoconference or in person.This helps to ensure that studentsare well prepared for the program,and that MSA staff are invested inthe success of the students.Once admitted, students havemany opportunities for academichelp and performance coaching.MSA Staff share a deepcommitment to student success.AREAS FOR IMPROVEMENT:We have found that applicantssometimes share on the Internetthe technical questions we ask offinalists. Improving our interviewstrategy, and personalizingquestions to the applicant willhelp insure that candidates aretruly prepared to enter theprogram.

We have changed the format ofapplicant interviews to be moreindividual to the applicant,focusing on each individual'sacademic and professionalexperiences, to better determinetheir level of preparation for theprogram. We have created twonew staff positions to meet thedemands of increased enrollmentin the next academic year, andare currently recruiting. We havealso decided to hire two additionalteaching assistants to workdirectly with students requiringremedial help.

Meet admissions target for averageundergraduate GPA (UGPA) of 3.50or greater

Admissions statisticsFINDINGS:The averageundergraduate GPA of admittedstudents in the Class of 2015 was3.51. More than half (56%)previously graduated with honors.

STRENGTHS: A large applicantpool ensures selectivity whichhelps to further establish the MSAprogram as a selective program ofhigh quality. AREA FORIMPROVEMENT: Ongoing effortsto increase or at the very leastmaintain the number ofapplications will be critical asmore and more programs becomeavailable.

We worked with UNCWilmingtonto give an on campuspresentation to a large audienceof interested students. We havedecided to schedule on siteinformation session/recruitingevents at our largest "feeder"institutions in North CarolinaUNC Chapel Hill, UNCWilmington, and AppalachianState University. We expect toschedule additional recruitingvisits to other colleges anduniversities as time and budgetpermit.

Maintain an admissions profile(acceptance rate, enrollment rate,and UGPA) that is equivalent to, orbetter than comparablysized MBAprograms at public universitiesranked in the Top10

Admissions statisticsFINDINGS:The MSA programbenchmarks itself against thehighest ranked public universityMBA programs with similarlysized cohorts: Georgia Institute ofTechnology, Ohio StateUniversity, University ofWisconsinMadison, and ArizonaState University. The MSAsurpasses all four of theseprograms (all ranked 27 in the USNews 2014 rankings), inacceptance rate, enrollment rate,and undergraduate GPA foradmitted students.

STRENGTHS: Our reputation asthe nation's first and leadinganalytics program, our annualplacement statistics, theprogram's design and short timeto degree, and our practical andwholistic approach to professionaleducation are among thestrengths that makes uscompetitive with leadingcomparably sized MBA programs.

See above actions on relatedoutcomes.

Objective 2: To achieve an excellent placement of graduates and a return on investment for the analytics degree




Maintain job placement outcomes(placement rates and salaries) that

Placement statisticsFINDINGS:The MSA program

We have decided to focus ourcompany specific networking

are equivalent to, or better thancomparablysized MBA programs atpublic universities ranked in the Top10.

benchmarks itself against thehighest ranked public universityMBA programs with similarlysized cohorts: Georgia Institute ofTechnology, Ohio StateUniversity, University ofWisconsinMadison, University ofMinnesota, and Arizona StateUniversity. The MSA ranked thirdwhen compared with these fiveinstitutions, with an average basesalary offer of over $100,000. TheMSA led all five institutions inplacement rate, with 100$placement by graduation,compared to rates between 85and 65 percent for the otheruniversities.

STRENGTHS: MSA studentscontinue to be in demand with awidening array of employers inmultiple industry sectors. AREAFOR IMPROVEMENT: With theexpansion of the program to 115students, we need to reach out tomore potential employers, as wellas increase opportunity forstudent contact with thoseemployers likely to make multipleoffers.

events on employers likely tomake multiple offers, and toimplement separate networkingevents for groups of smalleremployers.

Maintain job placement outcomes(placement rates and salaries) thatare equivalent to, or better thancomparablysized, 1year quantbased MS programs at fourbenchmark schools (Berkeley, MIT,Cornell and Carnegie Mellon

Placement statisticsFINDINGS:The MSA programcompared favorably with the fourbenchmark schools. The MSA jobplacement rates with 6090 daysof graduation were higher thanCarnegie Mellon's Master ofInformation SystemsManagement, MIT's Master ofFinance, UC Berkeley's Master ofFinancial Engineering, andCornell's M.Engineering inOperations Research andInformation Engineering. TheMSA average base salary washigher than all institutions otherthan the UC Berkeley MFE.

STRENGTHS: The MSAcompares well as an investmentwith prestigious and establishedprograms with much higher tuitioncosts.

No actions warranted at this time.

Attain a job placement rate of 90percent or higher by graduation

Placement statisticsFINDINGS:We again acheived100% placement by graduation forstudents, thus exceeding ourgoal.

STRENGTH: We have enjoyed astable cadre of employers whoreturn each year, often bringingprogram alumni with them, torecruit and hire students. AREAFOR IMPROVEMENT: With theexpansion of the program to 115students, we need to reach out tomore potential employers, as wellas increase opportunity forstudent contact with thoseemployers likely to make multipleoffers.

We have decided to focus ourcompany specific networkingevents on employers likely tomake multiple offers, and toimplement separate networkingevents for groups of smalleremployers.

Attain a average ROI payback periodof less than 36 months

ROI statistics FINDINGS:ROImeasures the length of time aftercompleting the degree it takes theaverage student to recover thecost of attending (tuition and fees)and the year of lost salary (preMSA salary). Based on a surveyof MSA students (95% responserate), the payback period for NCresidents averaged 19.7 months,and for nonresidents, 20.4months. The net 3year ROI forClass of 2015 was $128,500 forNC residents, and $155,500 fornonresidents.

STRENGTHS: The program is agood investment for studentsattending the program, and iscompetitively priced relative toother programs. Salaries offeredto MSA students are competitivewith programs that costsignificantly more to attend.

No changes warranted at thistime.

Attain an average ROI paybackperiod that is better (shorter) thancomparablysized MBA programs atpublic universities ranked in the Top10

ROI statistics FINDINGS:TheMSA program benchmarks itselfagainst the highest ranked publicuniversity MBA programs withsimilarly sized cohorts: GeorgiaInstitute of Technology, OhioState University, University ofWisconsinMadison, University ofMinnesota, and Arizona StateUniversity. The ROI paybackperiod for the MSA isconsiderably shorter than that ofall of the other five institutions

STRENGTHS: The return oninvestment for the MSA comparesfavorably with established andhighly ranked MBA programs andis an excellent educational value.

No actions warranted at this time.

Attain a ratio of job offers percandidate of 2.0 or greater

Job offer statisticsFINDINGS:MSA students in theClass of 2015 received onaverage three job offers.



Attain a ratio of job interviews percandidate of 10.0 or greater

Job interview dataFINDINGS:MSA students in theClass of 2015 had, on average,12 initial interviews.



Objective 3: To maintain the program as effective, efficient, and competitive with similar programs




Maintain the program as a selfcontained and sustainable businessmodel (i.e. operate within thebudgetary cost constraints providedby tuition revenue generated for theprogram)

Budget statistics FINDINGS: Theprogram operated within itsbudget with the exception of aonetime infusion of $600,000 infunding from the Provost's officeto provide infrastructure for,furnish, and relocate faculty andstaff to a new building toaccommodate the addition of anadditional 30 students to theprogram.

STRENGTHS: The Institute isfinancially sustainable withenrollment increase funding,summer session tuition andpremium tuition. AREAS FORIMPROVEMENT: The Instituteneeds to raise additional nonrestricted gift funding to supportprograms not appropriate for statefunds.

We have decided to moreassertively pursue fundingopportunities with potentialemployers.

Maintain resident and nonresidenttuition at or below the average forother similar MS degree programs

Tuition comparisons FINDINGS:The MSA tuition remains modest,well below most other full timeanalytics degree programs.In acomparison of 28 full timeanalytics programs, the MSA is inthe bottom quartile of tuitioncosts.

STRENGTH: The MSA is anexcellent educational value. Foran investment far less than thetuition of most fulltime analyticsprograms, MSA students receivean excellent educationalexperience, with job prospectsand earnings unparalleled by otherprograms. AREA FORIMPROVEMENT: We need tobetter inform prospective studentshow our program differs from ourcompetitors, what our tuitioncovers, and the outcomes wedeliver for their tuition dollars.

We have decided to create aninformational piece that highlightsthe differences between ourprogram and others so thatcandidates have the data theyneed to make informed decisionsif offered admission.

Objective 4: Data mining and machine learning




Graduates should be able toeffectively use the SAS EnterpriseMiner interface.

Homework, Test Results,Evaluation RubricFINDINGS:94% of studentssuccessfully passed PredictiveModeler Using SAS EnterpriseMiner 7 certification

STRENGTH: The program iseffective in enabling students touse SAS Enterprise Minerinterface. 94% passing rate onPredictive Modeler certification isexcellent. High certification ratesare a strength of the MSAprogram. AREA FORIMPROVEMENT: Increase thenumber of testing opportunities toallow more flexibility, and inparticular to enable students tocertify prior to the start ofemployment interviews

We have decided to addopportunities for PredictiveModeler certification prior to theJanuary/February recruiting timeframe.

Graduates should be able torecognize and develop associationand sequence analyses.

Homework, Test Results,Evaluation Rubric FINDINGS:Tested by exam (avg. score 88.6)and homework 2 (avg score.79)Sequence analysis, a minormodification, covered in classdiscussion.

STRENGTH: Test scores showinstruction in the program isstrong. AREA FORIMPROVEMENT: Sequenceanalysis requires more detailedcoverage, as several practicumteams struggled with this concept

We have decided to place equalemphasis on Association andSequence Analysis so thatstudents clearly understand thedifference and when eachtechnique is appropriate.

Graduates should be able to explainthe role of statistical tests in formingdecision trees.

Homework, Test Results,Evaluation RubricFINDINGS:Tested by exam (avg.score 88.6) and homework 4 (avg.score 80).

STRENGTH: Test scores showthat the program is effective inenabling students to achieve thisoutcome. AREA FORIMPROVEMENT: The loweraverage score on the homeworksuggests the program needsimprovement in this area.Instruction could be more focusedand/or materials provided whichoutline major points covered.

We have decided to reviewhomework assignment foralignment with instruction andincrease teaching assistantavailability for this topic. We havealso decided to provide additionalresources for students to retainmajor ideas regarding decisiontree outputs, e.g. a list ofvocabulary with definitions andexamples.

Graduates should be able to fitdecision trees to binary data andinterpret the results.

Homework, Test Results,Evaluation Rubric FINDINGS:Tested by exam (avg. score 88.6)and homework 4 (avg. score 80).

We have decided to combinemodules so that topics such asfitting models to binary data andinterpreting the result are

AREA FOR IMPROVEMENT:Repetitive content in Data Mining1 and Logistic Regression. Both,fundamental and important, wereoccasionally covering the samematerial.

addressed only once. The newBinary Response Analyticsmodule more concisely teachestopics currently covered in DataMining 1 and Logistic Regression.

Graduates should be able to fitregression trees.

Homework, Test Results,Evaluation Rubric FINDINGS:Regression covered in homework1 (avg score 79), regression treescovered by in classdemonstrations (all questionsanswered)

STRENGTH: Classroominstruction for this topic in theprogram is strong. AREA FORIMPROVEMENT: Could improvehomework performance

We have decided to reviewhomework assignment foralignment with instruction andincrease teaching assistantavailability for this topic

Graduates should be able to explaindiscriminant analysis.

Homework, Test Results,Evaluation Rubric FINDINGS:Extensive notes on discriminantcovered in class, all questionsanswered. In class demo andknowledge tested on final exam(avg. score 92.6) and inhomework 6 (avg score 86) alongwith cluster analysis.

STRENGTH: The programprovides excellent coverage ofthis topic demonstrated in examand homework scores. AREAFOR IMPROVEMENT: None


Graduates should be able torecognize and interpret regression forbinary responses.

Homework, Test Results,Evaluation RubricFINDINGS:Logistic regressioncovered in homework 5 (avgscore 91) and on exam (avg.score 92.6).

AREA FOR IMPROVEMENT:Repetitive content in Data Mining1 and Logistic Regression. Both,fundamental and important, wereoccasionally covering the samematerial.

We have decided to combinemodules so that topics such asfitting models to binary data andinterpreting the result areaddressed only once and toimplement a Binary ResponseAnalytics module to moreconcisely teach topics currentlycovered in Data Mining 1 andLogistic Regression.

Graduates should be able to dividedata using clustering techniques.

Homework, Test Results,Evaluation RubricFINDINGS:Covered by in classdemos and tested in homework 6(avg. score 91)

STRENGTH: The programprovided good coverage of thistopic demonstrated in examscores. AREA FORIMPROVEMENT: Students coulduse more of the linear algebrabuildup that has already beentaught for this topic, as mostclustering tools are described inthe literature using linear algebranotation.

We will increase emphasis onlinear algebra moduleantecedents.

Graduates should be able to accountfor oversampling, profits, and losses.

Homework, Test Results,Evaluation RubricFINDINGS:Covered by in classdemos.

AREA FOR IMPROVEMENT:The program is not as effective asit should be in enabling studentsto account for oversampling,profits, and losses Covered by inclass demonstrations. A crucialtopic for practicum projects,students appear to need a morethorough stepbystepdemonstration, and learningoutcome assessment withhomework or testing.

We have decided to improve thecurrent demonstration byproviding a more thorough stepbystep demonstration. We willimprove learning outcomeassessment with homework ortesting.

Graduates should be able to identifyand use techniques for choosing thebest model from many.

Homework, Test Results,Evaluation Rubric FINDINGS:Covered by in class demos.

AREA FOR IMPROVEMENT:The program did not measurewhat students learned from theclass demonstration.

We have decided to improvelearning outcome assessmentwith homework or testing.

Graduates should be able to buildmodels using variable selectiontechniques.

Homework, Test Results,Evaluation RubricFINDINGS:Covered by in classdemos.

AREA FOR IMPROVEMENT:The program did not measurewhat students learned from theclass demo.

We have decided to improvelearning outcome assessmentwith homework or testing.

Graduates should be able torecognize and build neural networks.

Homework, Test Results,Evaluation RubricFINDINGS:Covered by in classdemos and exam (avg. score92.6).

STRENGTH: Excellent coverageof this topic demonstrated inexam scores (avg. score 92.6).AREA FOR IMPROVEMENT:We need to provide additionalinformation about responsesurface and why a neural networkis so different from any of theother binaryresponse tools thatare taught. This is a good place tointroduce the idea and how tothink about those concepts aspart of the modelling process.

We will add content on responsesurface, and the distinctionbetween neural networks andother binary response tools.

Objective 5: Data visualization




Graduates should be able to describein a way that demonstrates generalunderstanding, the main goals of datavisualization.

Homework, Test Results,Evaluation Rubric FINDINGS:Allof these related objectives weremet through the team homeworkproject using Tableau. Eachhomework team was given realdata and a set of five tasks tocomplete from an externalcompany, Plexis Corp. The teamswere required to use Tableau tobuild visualizations to completethe tasks, then present the resultsto a member from Plexis'sTableau Analytics groups andPlexis's CEO. All groups receivedeither check or checkplus,passing grades, demonstratingtheir ability to use Tableau topresent data in the standard chartand map formats listed above.Different groups chose to usedifferent subsets of the charts andmaps, making appropriatedecisions that demonstrated theirability to understand the strengthsand limitations of each chart andmap type to a given task.

STRENGTH: Instruction andsupportingdocumentation/webpage wasstrong. Outcomes measured byhomework and projectperformance indicate students areadept at visualizations.

Because demand for visualizationtopics is high, and critical, wehave decided to pursue creating afull time faculty position in datavisualization.

Graduates should be able torecognize the main types of chartsand plots (line, bar, pie, scatter, etc.).

Homework, Test Results,Evaluation Rubric FINDINGS:See above summary.

STRENGTH: Instruction andsupporting documentation andwebpage was strong. Outcomesmeasured by homework andproject performance indicatestudents are adept atvisualizations.


Graduates should be able to Selectthe type of chart or plot best suited toa particular type of data andvisualization goal.

Homework, Test Results,Evaluation Rubric FINDINGS:Seeabove summary.



Graduates should be able torecognize the main types of maps(choropleth, contour, dot, dasymetric,etc.).




Graduates should be able to selectthe type of map best suited to aparticular type of data andvisualization goal.


Instruction and supportingdocumentation and webpage wasstrong. Outcomes measured byhomework and projectperformance indicate students areadept at visualizations.


Objective 6: Text Analytics




Graduates should be able to describein a way that demonstrates generalunderstanding the main goals of textanalytics and text mining.

Homework, Test Results,Evaluation Rubric FINDINGS:Allof these related objectives weremet through the team homeworkproject, where student groupswere asked to choose a textdomain and task, analyze textfrom the domain, then presenttheir results to their classmates.All groups received either checkor checkplus, passing grades,demonstrating their ability to usetheir knowledge of text analyticsto analyze and present text datain insightful and useful ways.Different groups chose to usedifferent text analytics

STRENGTH: Current instructionis strong as homework outcomesand project performance show.

No changes to the curriculumwarranted at this time, butbecause demand for visualizationtopics is high, and critical, wehave decided to pursue creating afull time faculty position in datavisualization.

approaches, making appropriatedecisions that demonstrated theirability to understand the strengthsand limitations of each approachto a given task. Evidence ofvisualization knowledge wasfurther demonstrated during thepracticum presentations, where asubset of the practicum groupsused text analytics as part of theiranalysis of practicum data.Feedback from the facultyevaluating the practicumshighlighted the students' ability tocorrectly recognize the value oftext analytics in the situationswhere they chose to use it, and inproviding useful feedback on theraw text they analyzed

Graduates should be able to recallthe strengths and limitations ofdifferent methods of systematicallyrepresenting text and understand howto apply these methods to a textcorpus.


STRENGTH: Current instructionis strong as homework outcomesand project performance show.

No changes to the curriculumwarranted at this time.

Graduates should be able to identifydifferent approaches to computingtext similarity, and how to use thesemeasures to organize text based onsimilarity clustering.


STRENGTH: Current instructionis strong as homework outcomesand project performance show.AREA FOR IMPROVEMENT:Include additional application ofPython to the manipulation of textdata.

We have added a Pythonworkshop that includes TextMining to the curriculum.

Graduates should be able torecognize and describe differentmodels of emotion or sentiment.


STRENGTH: Current instructionis strong as homework outcomesand project performance show.AREA FOR IMPROVEMENT:Include sentiment analysissoftware.

We have upgraded our currentsoftware to include a sentimentanalysis component

Graduates should be able to employdifferent approaches to usingemotional models to estimate andrepresent sentiment contained in atext corpus.


STRENGTH: Current instructionis strong as homework outcomesand project performance show.AREA FOR IMPROVEMENT: Asnoted above, include sentimentanalysis software.

We have upgraded our currentsoftware to include a sentimentanalysis component

Objective 7: Python




Graduates should be able todemonstrate a basic understanding ofcomputer programming with acommon procedural programminglanguage.

Homework, Test Results,Evaluation Rubric FINDINGS: Allof these related objectives weremet through the inclassexercises, where students wereasked to solve simpleprogramming problems that testedtheir knowledge of basic conceptslike use of variables, basicprogram flow, conditionals,functions, and analytics usingnumpy and pandas. Theexercises were not formallymarked, instead, the faculty andstaff evaluated the students'abilities inclass as they workedon the exercises. Given the broadbackgrounds of the students,faculty and staff focused onstudents with no prior backgroundin programming. All studentssuccessfully completed the inclass exercises. Evidence ofprogramming knowledge wasfurther demonstrated during thepracticum presentations, where alarge subset of the practicumgroups used programming invarious ways as part of theiranalysis of practicum data. Thisincluded programming in Python,Visual Basic, R, SAS, andTableau. Feedback from thefaculty evaluating the practicumsvalidated the students ability toleverage their programmingknowledge to manage, analyze,and present raw data included as

STRENGTH:Python instructionwas improved over the last yearin organization, quantity (therewas more), timing (it came soonerin the curriculum) and moduleresources such as the classwebpage. We also supported astudent led Interest Group thatfocused on the use of Python. Inparticular, that group, along with afaculty mentor, was able to showhow Python could be used indifferent areas of our curriculum.It is a strength of the MSAprogram, that students areexposed to the basic functionalityof Python and also supported ifthey have personal or careergoals to learn more.

We have decided to support theuse of Python in other areas ofthe curriculum currently focusedon different software such asSAS & R. We will encourage andsupport a student led specialinterest group in Python everyyear.

part of their practicum projects.NLTK WAS NOT TAUGHT thisyear.

Graduates should be able to design,implement, and test small programswritten in Python.


See above comments. See above comments.

Graduates should be able to performbasic analytic operations with Pythonusing common external libraries (nltk,numpy, pandas, etc.).


See above comments. See above comments.

Objective 8: Customer segmentation and positioning




Graduates should be able torecognize real world applications ofsegmentation theory.

Homework, Test ResultsFINDINGS:A homeworkassignment and a final project onsegmentation was given to 100%of the students on real worldindustry data (in teams). The finalsegmentation project involvedbusiness reports and presentationby each student teams. 80% ofthe students performedsatisfactorily in the HWassignment and 95% of thestudents performed satisfactorilyin the segmentation project.

STRENGTH: Homeworkassignment mimics real workexperience in that it requires thatstudent work in teams, submit abusiness report, and practicepresentation skills in deliveringfindings.


Graduates should be able to employdifferent techniques and methods forsegmenting various types of datausing different statistical software(SAS, SPSS, R).

Homework, Test ResultsFINDINGS:One homeworkassignment and a final projectmeasured this outcome for allstudents. The homeworkassignment required students tosubmit a business report and thefinal project required the studentsto submit a report and make a 20minute presentation to the staff ofIAA. 75% of the studentssatisfactorily completed thisoutcome.

STRENGTH: Homework mimicsreal work experience in that itrequires that student work inteams, submit a business report,and practice presentation skills indelivering findings. AREA FORIMPROVEMENT: We are seeingmore practicum projects withdichotomous and categorical dataand students struggling with thesegmentation of that kind of data.Currently, the class only showshow to cluster that type of data ina couple of slides but in the futurewe need to include a homeworkproblem to address it. In addition,the students are also being askedto use R for segmentation. Weneed to guide them towardsresources that show how to useR or show a small example in theclass.

A homework problem has beencreated with real data fromindustry with dichotomous andcategorical data (mix) to segmentcustomers. Students will besolving the problem in teams.Secondly, the same examplesolved in the class using SASand SPSS will be solved using R.

Graduates should be able to applydifferent techniques for variablereduction including principalcomponents, common factoranalysis, etc.

Homework, Test ResultsFINDINGS:One homeworkassignment was given to all thestudents with real data to applydimension reduction techniquesand submit a business report forgrading. 100% of the studentscompleted the assignment andmet the objectives satisfactorily.

STRENGTH: Homework mimicsreal work experience in that itrequires that student work inteams, submit a business report,and practice presentation skills indelivering findings.


Graduates should be able to solve asegmentation problem usingstatistical software (SAS, SPSS, R)and real world data.

Homework, Test ResultsFINDINGS:Not measured thisyear.

AREA FOR IMPROVEMENT:The program did not measurewhether students were able tosolve a segmentation problemusing actual data and threestatistical software tools.

A homework problem has beencreated with real data fromindustry with dichotomous andcategorical data (mix) to segmentcustomers. Students will besolving the problem in teams.Secondly, the same examplesolved in the class using SASand SPSS will be solved using R

Objective 9: Design of experiments




Graduates should be able to explainthe concept of designing experimentsand its applications beyondlaboratories in a business setting.

Homework, Test ResultsFINDINGS: All students had tocomplete three homeworkassignments which measured thisoutcome using real industry data.The students worked in teamsand submitted business reportsfor each of the assignments withtheir recommendations andanalysis. About 80% of thestudent teams met this objective

STRENGTHS: Design ofexperiment topics are taughtkeeping the practical applicationsin mind and thinking about thedigital future (testing andtargeting). Homework mimics realworld experience in directmarketing and requires studentsto work in teams and submit abusiness report.


satisfactorily through thesereports.

Graduates should be able to controland vary parameters to get thedesired outcome in direct marketingexperiments.

Homework, Test ResultsFINDINGS: Two homeworkassignments in this modulerequired students to vary severalparameters of the data and reportresults in business reports. 80%of the students met this objective.

STRENGTHS: Students areprovided with a text withnumerous examples of varyingparameters of the data especially for those models thatcannot be included in the classlectures for self study. AREAFOR IMPROVEMENT: Theprogram did not assess thelearning outcomes for thismaterial. In the future, a fewquestions should be included atthe end of the text or in moodlefor students to answer so theinstructor knows that the studentslooked at the supplementalmaterial.

A ten question quiz has beenincluded in the moodle classwebsite and graded to evaluatethe student's understanding of thecontent.

Graduates should be able to useSAS and JMP software and sampleindustry data to demonstrateunderstanding of techniques used indesigning and analyzing experiments.

Homework, Test ResultsFINDINGS:100% of the studentsmet this outcome through threeHW assignments that requiredstudents to use SAS software orJMP software. Each student teamsubmitted a business report thatwas graded to meet thisobjective/outcome.

STRENGTH: Homework mimicsreal world experience in design ofexperiments using SAS or JMP (asoftware of their choice).


Objective 10: Marketing mix and web analytics




Graduates should be able to identifydifferent marketing mix models andmarket basket models for use inbusiness settings.

Homework, Test ResultsFINDINGS:All the students hadto complete three homeworkassignments, one on marketbasket model, one on webanalytics and one on customer liftime value to measure thisoutcome. Each of theassignments required students todeliver a business report in teamsfor grading. 95% of the studentteams were able to satisfy theabove outcome.

STRENGTH: The programincludes mixed model analysis asa regular component. AREA FORIMPROVEMENT: This modulehas typically been scheduled afteremployer interviews, whichfrequently include questions onmixed model analysis.

We have scheduled thisinstruction in December, inadvance of interviews.

Graduates should be able to developa pricing model using customer datato demonstrate understanding ofmarketing mix and market basketmodels.

Homework, Test ResultsFINDINGS:Not Measured thisreporting period.

STRENGTH: Despite lack of timeto cover the material in class,supplementary materials coveringpricing materials were provided toall students. AREA FORIMPROVEMENT: The programdid not measure the outcomes forthis area.

A four question quiz has beenadded to Moodle for the studentsto complete after going throughthis supplemental material.

Graduates should be able to applymarketbased models to providerecommendations to the customer forproduct, promotion and pricingchanges to its offerings.

Homework, Test ResultsFINDINGS: All students weregiven an assignment on marketbasket analysis to measure theoutcome. The students submitteda business report with theirrecommendations for grading.85% of students were able tomeet the objective satisfactorily.

STRENGTH: The programthoroughly exposes the studentsto different applications of marketbasket analysis. The models inthis class are an extension of themarket basket models taught indata mining by Dr. DaveDickey,and as well they focus ona different application in theindustry.

No action warranted at this time.

Graduates should be able torecognize web data reporting tools(e.g., Google Analytics), ad andcampaign testing tools (e.g.,optimizely), visualization tools for bigdata, and strategies for integratingweb and offline data.

Homework, Test ResultsFINDINGS: In class polls wereconducted with 5 questions onweb analytics tools to understandstudent understanding of theabove outcome. 90% of thestudents successfully respondedto the above poll.

STRENGTH: Students areinstructed on Google Analytics asa reporting tool and have theoption of pursuing GoogleAnalytics certification at noadditional cost.

We have decided to include theGoogle Analytics premium dataanalytics parameters in ourcoverage.

Graduates should be able tocomplete Google Analyticscertification to demonstrateunderstanding of web analyticsbasics and applications.

Homework, Test ResultsFINDINGS:All students receivedinstruction. Those who wereconsidering careers that wouldinclude a web analyticscomponent were encouraged tosit for the Google Analyticscertification exam. Of the entireclass (not all of whom pursue webanalytics careers) 60% ofstudents passed the optionalGoogle Analytics certification.

STRENGTH: The programprovides valuable industrycertification opportunities forstudents. AREA FORIMPROVEMENT: Not allstudents pursue the optionalGoogle Analytics certification andthus we have less than completedata to assess this as a learningoutcome.

We have decided to stronglyencourage students to add thiscertification to their toolbox. Anoptional 90 minute class onAdWords use in advertising willbe included for studentsinterested in pursuing thisadditional certification.

Objective 11: Logistic regression




Graduates should be able to identifythe key differences between logisticregression and linear regression.

Homework, Test ResultsFINDINGS:A homeworkassignment and a final projectwas given to 100% of thestudents on real world industrydata (in teams). The final projectinvolved business reports by eachstudent team. 93% of thestudents performed satisfactorilyin the HW assignment and 88%of the students performedsatisfactorily in the project.

STRENGTH: Logistic regressionis well covered in both the logisticregression class and Data Mining1. AREA FOR IMPROVEMENT:This topic is covered in multiplemodules and can probably besimplified.

We have decided to removelogistic regression from the DataMining 1 module.

Graduates should be able todistinguish between nominal andordinal variables and the differentstatistical tests between them.

Homework, Test ResultsFINDINGS:A homeworkassignment was given to 100% ofthe students on simulatedindustry data (in teams). The HWinvolved business reports by eachstudent team. 77% of thestudents performed satisfactorilyon this portion of the HWassignment.



Graduates should be able to buildlogistic regression models (in all theirforms – binary, ordinal, nominal)using the statistical softwarepackages SAS and R.




Graduates should be able to interpretthe output from logistic regressionmodels (in all their forms – binary,ordinal, nominal) using the statisticalsoftware packages SAS and R.

Homework, Test ResultsFINDINGS: A homeworkassignment and a final projectwas given to 100% of thestudents on real world industrydata (in teams). The final projectinvolved business reports by eachstudent team. 71% of thestudents performed satisfactorilyin the HW assignment and 94%of the students performedsatisfactorily in the project.



Graduates should be able to interpretthe meaning of odds ratios.

Homework, Test ResultsFINDINGS: A homeworkassignment was given to 100% ofthe students on simulatedindustry data (in teams). The HWinvolved business reports by eachstudent team. 88% of thestudents performed satisfactorilyon this portion of the HWassignment.



Objective 12: Times series and forecasting




Graduates should be able todecompose a time series into itsthree basic components – trend,seasonality, and remainder.

Homework, Test ResultsFINDINGS:A homeworkassignment was given to 100% ofthe students on industry data (inteams). The HW involvedbusiness reports by each studentteam. 100% of the studentsperformed satisfactorily on thisportion of the HW assignment.

STRENGTH: Student outcomesimproved with the new of materialpresenation for the time seriesmodule. AREA FORIMPROVEMENT: Because thiswas the first year of teaching thismaterial in the new order, not allmaterial was covered.

We have added a lecture to themodule in order to cover allrelevant material.

Graduates should be able todistinguish between the threedifferent correlation functions – ACF,PACF, and IACF.

Homework, Test ResultsFINDINGS:A homeworkassignment was given to 100% ofthe students on simulatedindustry data (in teams). The HWinvolved business reports by eachstudent team. 76% of thestudents performed satisfactorilyon this portion of the HWassignment.



Graduates should be able to explain Homework, Test Results We have added a lecture to the

the difference between a stationaryand nonstationary time series.

FINDINGS: A homeworkassignment was given to 100% ofthe students on simulatedindustry data (in teams). The HWinvolved business reports by eachstudent team. 83% of thestudents performed satisfactorilyon this portion of the HWassignment.


module in order to cover allrelevant material.

Graduates should be able to build thedifferent classes of time seriesmodels (Exponential Smoothing,ARIMA, and Neural Network) usingthe statistical software packagesSAS and R.

Homework, Test ResultsFINDINGS: A homeworkassignment and a final projectwas given to 100% of thestudents on real world industrydata (in teams). The final projectinvolved business reports by eachstudent team. 94% of thestudents performed satisfactorilyin the HW assignment and 100%of the students performedsatisfactorily in the project.



Graduates should be able to interpretthe output of different classes of timeseries models (ExponentialSmoothing, ARIMA, and NeuralNetwork) using the statisticalsoftware packages SAS and R.




Graduates should be able todiagnose different classes of timeseries models for accuracy andreliability.




Graduates should be able to forecastdifferent types of time series models.




Objective 13: Survival analysis




Graduates should be able to buildsurvival curves using both commontechniques – KaplanMeier and LifeTable.


STRENGTH: The modulepresents a blend of evaluationtechniques, including projects,homework assignments, andassessments. AREA FORIMPROVEMENT: Add businessoriented data set examples, as allcurrent examples are based in thesocial sciences.

We have collected data sets withmore business appeal in order toincrease the variety of examplesand relevance for students

Graduates should be able to interpretthe survival and hazard probability ofa data series.




Graduates should be able to design adata set that contains both censoredand uncensored observations.

Homework, Test ResultsFINDINGS:A homeworkassignment was given to 100% ofthe students on industry data (inteams). The HW involved


business reports by each studentteam. 100% of the studentsperformed satisfactorily on thisportion of the HW assignment.


Graduates should be able to build thedifferent classes of survival analysismodels (Accelerated Failure Time,Cox Regression).




Graduates should be able to identifythe proper distributional assumptionin an Accelerated Failure Timemodel.




Graduates should be able to identifycases where competing risks areoccurring.

Homework, Test ResultsFINDINGS: A homeworkassignment was given to 100% ofthe students on industry data (inteams). The HW involvedbusiness reports by each studentteam. 94% of the studentsperformed satisfactorily on thisportion of the HW assignment.



Objective 14: Exploratory data analytics/fraud detection




Graduates should be able to buildand analyze a social network dataset.

Homework, Test ResultsFINDINGS: All of these relatedoutcomes were met by an inclass assignment given to 100%of the students (in teams) onsimulated industry data whichtested their ability to build andanalyze social network data,identify subgroups, centers,closeness, brokers, etc, transformtransactional data into a usableformat and identify commoncharacteristics of fraud. 100% ofthe students performedsatisfactorily on this portion of theassignment.

STRENGTH: Students workthough a complex andcomprehensive problem thatutilizes all of the topics presentedin module. AREA FORIMPROVEMENT: More time forstudents to solve the problem isneeded.

We added a lecture period to themodule to provide more inclasstime for teams to develop asolution to the assigned problem.

Graduates should be able to identifysubgroups, centers, closeness,brokers, bridges, diffusion, andadoption in a social network.

Homework, Test ResultsFINDINGS: See above summary

STRENGTH: Students workthough a complex andcomprehensive problem thatutilizes all of the topics presentedin module. AREA FORIMPROVEMENT: More time forstudents to solve the problem isneeded.


Graduates should be able totransform transactional data into ausable format for typical forms ofanalysis.

Homework, Test ResultsFINDINGS:See above summary

STRENGTH: Students workthough a complex andcomprehensive problem thatutilizes all of the topics presentedin module. AREA FORIMPROVEMENT: More time forstudents to solve the problem isneeded. We added a lectureperiod to the module to providemore inclass time for teams todevelop a solution to the assignedproblem.


Graduates should be able to identifycommon characteristics of fraud inthe insurance industry.

Homework, Test ResultsFINDINGS: See above summary


STRENGTH: Students workthough a complex andcomprehensive problem thatutilizes all of the topics presentedin module. AREA FORIMPROVEMENT: More time forstudents to solve the problem isneeded. We added a lectureperiod to the module to providemore inclass time for teams todevelop a solution to the assignedproblem.

Objective 15: Optimization




Graduates should be able to applyinteger and mixedintegeroptimization techniques to identifythe product mix and transportationschedules that maximize profitabilitywhile satisfying demand constraints,capacity constraints andtransportation cost caps.

Homework, Test ResultsFINDINGS:Tested in Homework1 & 2 where, for example,students determined an optimalproduction plan for a single day ofoil production given multiplebrands and differing viscositystandards, as well as an optimaloutput plan for a companyproducing table lamps, floorlamps, ceiling fixtures, andpendant lamps with differing costsand constraints. 100% of studentssuccessfully completed thisassignment meeting or exceedingreporting requirements

Dr. Konstantinos Kyriakoulis, thefaculty member responsible forthis area of the curriculum, diedsuddenly on May 1, 2015. We arethus unable to address thisaspect of the review.

We have decided to identify newand existing faculty to teach thissubject area and develop newbenchmarks for future evaluation.

Graduates should be able to performData Envelopment Analysis toidentify the most efficient unit from aset of many candidates who producedifferent final products using differentinputs mix.

Homework, Test ResultsFINDINGS:Tested in Homework1 & 2 where, for example,students determined an optimalproduction plan for a single day ofoil production given multiplebrands and differing viscositystandards, as well as an optimaloutput plan for a companyproducing table lamps, floorlamps, ceiling fixtures, andpendant lamps with differing costsand constraints. 100% of studentssuccessfully completed thisassignment meeting or exceedingreporting requirements



Graduates should be able to identifythe optimal stockportfolio allocationthat minimizes risk and at the sametime achieves a target return.

Homework, Test ResultsFINDINGS:Taught (Topic 5:Nonlinear Optimization (Powell &Baker: Ch. 10) : SAS Code &Excel code / Portfolio optimizationusing SAS / Portfolio optimizationand efficient frontier SP100 data),but not tested.



Graduates should be able to usenonlinear optimization techniques(Gauss, GaussNewton, NewtonRaphson) to fit the best nonlinearmodel in a set of data.

Homework, Test ResultsFINDINGS:Taught (Topic 5:Nonlinear Optimization (Powell &Baker: Ch. 10) : SAS Code &Excel code / Portfolio optimizationusing SAS / Portfolio optimizationand efficient frontier SP100 data),but not tested.



Objective 16: Simulation and risk analysis




Graduates should be able to verifythe properties of statistical models bysimulating their behavior and identifythe impact of violating key modelingassumptions.

Homework, Test ResultsFINDINGS:Demonstrated inHomework 1 using Monte CarloSimulation as a device to verifythe properties of a modelor atheory. 100% of student teamssuccessfully completedassignment.



Graduates should be able to use theexpected value approach to calculatethe expected net present value andpotential losses for an investmentthat runs across multiple periods.

Homework, Test ResultsFINDINGS:Demonstrated inHomework 1 & 2 (build a MonteCarlo simulation to analyze NetPresent value of the expectedreturns) 100% of student teamssuccessfully completed HW1 & 2assignment meeting or exceeding



reporting requirements

Graduates should be able to utilizethe KolmogorovSmirnov, AndersonDarling and other nonparametricstatistics to identify and fit theappropriate distribution of differentreal datasets (e.g. oil prices acrosstime, daily oil production, leasecosts).

Homework, Test ResultsFINDINGS:Demonstrated in TermProject

Dr. Konstantinos Kyriakoulis, thefaculty member responsible forthis area of the curriculum, diedsuddenly on May 1, 2015. We arethus unable to address thisaspect of the review. We willidentify new and existing facultyto teach this subject area anddevelop new benchmarks forfuture evaluation.

Dr. Konstantinos Kyriakoulis, thefaculty member responsible forthis area of the curriculum, diedsuddenly on May 1, 2015. We arethus unable to address thisaspect of the review. We willidentify new and existing facultyto teach this subject area anddevelop new benchmarks forfuture evaluation.

Graduates should be able to performscenario and sensitivity analysis toidentify the most sensitivedecision/control variables in aproject.

Homework, Test ResultsFINDINGS:Demonstrated in termproject



Graduates should be able to usesimulation techniques to calculatethe expected Net Present Value,Value at Risk and Expected Shortfallof a project that extends acrossmany years; assess the risks andprovide recommendations for theirreduction.

Homework, Test ResultsFINDINGS:Demonstrated in termproject.



Graduates should be able to usesimulation to value real options, suchas "option to abandonproject"; identify the optimaloption's price and suggest alternativeoption contracts that minimize therisk of shareholders.

Homework, Test ResultsFINDINGS:Demonstrated in termproject.



Objective 17: Financial Analytics




Graduates should be able to evaluatethe performance of a portfolio throughthe usage of single factor models andbuild the optimal (risk vs return)portfolio allocation CAPM,portfolio's alpha, portfolio's beta.

Homework, Test ResultsFINDINGS:Case study onadvanced portfolio optimizationincluding Minimum variance usingGARCH VS. minimum CVaRportfolio. Duration: 1 lecture.Spring 2015.



Graduates should be able to analyzeportfolio risk and return from thestandpoint of a riskmanager, utilizinga timevarying beta estimation(EWMA).

Homework, Test ResultsFINDINGS:Case study: VaRcalculation using Monte Carlosimulation of different modelsincluding GARCH ; EWMA ;Factor models covered Spring2015



Graduates should be able to analyzethe timevarying volatility of aportfolio through the usage of ARCHand GARCH (symmetric andasymmetric) models.

Homework, Test ResultsFINDINGS:ARCH/GARCHModels with Reference materialand SAS code for inclassexamples (Effect of GARCHparamaters ; GARCH estimationof MSFT) covered spring 2015



Graduates should be able to buildcredit scorecards to rank differentcredit applicant; utilize clustering,decision trees and logistic regressionmodels.

Homework, Test ResultsFINDINGS:Students createdscorecards for evaluating all retailcredit applicants in homework 1,spring 2015. The Scorecardsassociated individual scores withprobability of default (PD). Cutoffpoints maintained the existingacceptance rate (75% ) whileminimizing event (default)ratesand/or Maintained the currentevent rate (2.5%) whilemaximizing the acceptance rateand Maximized the profitability ofthe department. 100% of thestudents completed theassignment successfully (inteams).



Graduates should be able to identifythe optimal cutoff point for ascorecard in a way that maximizesthe profitability of a credit institution.

Homework, Test ResultsFINDINGS:Students createdscorecards for evaluating all retailcredit applicants in homework 1,spring 2015. The Scorecardsassociated individual scores withprobability of default (PD). Cutoffpoints maintained the existingacceptance rate (75% ) while



minimizing event (default)ratesand/or Maintained the currentevent rate (2.5%) whilemaximizing the acceptance rateand Maximized the profitability ofthe department. 100% of thestudents completed theassignment successfully (inteams).

Graduates should be able to identifythe optimal cutoff point for ascorecard in a way that maximizesthe approved credit customers whilekeeping default rates the same.

Homework, Test ResultsFINDINGS:Homework, TestResults FINDINGS:Studentscreated scorecards for evaluatingall retail credit applicants inhomework 1, spring 2015. TheScorecards associated individualscores with probability of default(PD). Cutoff points maintained theexisting acceptance rate (75% )while minimizing event (default)ratesand/or Maintained the currentevent rate (2.5%) whilemaximizing the acceptance rateand Maximized the profitability ofthe department. 100% of thestudents completed theassignment successfully (inteams).



Objective 18: Linear algebra




Graduates should be able tomanipulate and simplify matrixequations using the properties ofmatrix multiplication, addition,inversion, transposition, symmetry.

Homework, Test ResultsFINDINGS:Tested in Quiz 1: Allstudents successfully passedQuiz 1 with a score of 80% orbetter. Also Tested in Final Exam all students passed the FinalAssessment with a score of 75%or better.

STRENGTHS: Quiz scores showthat students knew the notationand basics mathematics of linearalgebra well by the end of thecourse. AREAS FORIMPROVEMENT: If studentshave these basics mastered bythe time they begin this course,we can progress to more datacentric topics like PCA,correspondence analysis, markovchains, and multidimensionalscaling. We can provide morepractice on these early topics asthey are introduced so thatstudents do not fall behind.

Linear Algebra Primer has beenenhanced to include most of thisintroductory material so that it willnot be covered in depth by thiscourse.

Graduates should be able to computeand interpret common vector normsand similarity metrics.

Homework, Test ResultsFINDINGS: Tested in Quiz 1: Allstudents successfully passedQuiz 1 with a score of 80% orbetter. Also Tested in Final Exam all students passed the FinalAssessment with a score of 75%or better.

STRENGTHS: Quiz scores showthat students knew the notationand basics mathematics of linearalgebra well by the end of thecourse. AREAS FORIMPROVEMENT: If studentshave these basics mastered bythe time they begin this course,we can progress to more datacentric topics like PCA,correspondence analysis, markovchains, and multidimensionalscaling. Can provide morepractice on these early topics asthey are introduced so thatstudents do not fall behind.


Graduates should be able to solvesystems of equations using themethods of Gaussian Elimination andLeast Squares.

Homework, Test ResultsFINDINGS: Tested in Quiz 1: Allstudents successfully passedQuiz 1 with a score of 80% orbetter. Also Tested in Final Exam all students passed the FinalAssessment with a score of 75%or better.

STRENGTHS: Quiz scores showthat students knew the notationand basics mathematics of linearalgebra well by the end of thecourse. AREAS FORIMPROVEMENT: If studentshave these basics mastered bythe time they begin this course,we can progress to more datacentric topics like PCA,correspondence analysis, markovchains, and multidimensionalscaling. Can provide morepractice on these early topics asthey are introduced so thatstudents do not fall behind.


Graduates should be able to define(mathematically) and describe(geometrically) the notions of linearindependence, vector span, vectorspaces, basis vectors, eigenvalues,eigenvectors and projections.

Homework, Test ResultsFINDINGS: Tested in Quiz 1: Allstudents successfully passedQuiz 1 with a score of 80% orbetter. Also Tested in Final Exam all students passed the FinalAssessment with a score of 75%

STRENGTHS: Teaching theseconcepts by visual examplesworked well for most students.AREA FOR IMPROVEMENT:The instruction can be improvedby using more datadriven visualexamples.

We have included more datadriven visual examples in thisinstruction.

or better.

Graduates should be able to usesoftware to find eigenvalues andeigenvectors of a matrix.


STRENGTHS: Quiz resultsdemonstrated students' ability toperform this task, as well as theirunderstanding that differentsoftware packages may givedifferent output. AREA FORIMPROVEMENT: give moreexamples of conflicting outputbetween SAS and R, and havestudents equivocate the outputsby reversing the signs of eitherthe components or thecoordinates of PCA.

We have added more examples ofconflicting output between SASand R, and having studentsequivocate the outputs byreversing the signs of either thecomponents or the coordinates ofPCA.

Graduates should be able to applyPrincipal Components Analysis todata for clustering, variableclustering, dimension reduction, andbiased regression.

Homework, Test ResultsFINDINGS:Tested in Homework1: All students successfullycompleted Homework 1 with ascore of 90% or better. AlsoTested in Final Exam allstudents passed the FinalAssessment with a score of 75%or better.

STRENGTHS: Students likedbeing able to solve the problem inthe homework, as it was aproblem which complete baffledthem with their previous toolset(without PCA). AREA FORIMPROVEMENT: The inclassexamples were helpful, but used avery tired dataset which manystudents have seen. Manystudents also had trouble graspingthe vocabulary of PCA.

We have improved the module byusing new dataset fordemonstration of PCA inclass.We have created worksheets toguide them through newvocabulary, relating the new datadriven examples to the moretraditional mathclass examplesdone previously.

Graduates should be able todetermine when Biased Regressionis appropriate.

Homework, Test ResultsFINDINGS: Tested in Homework1: All students successfullycompleted Homework 1 with ascore of 90% or better. AlsoTested in Final Exam allstudents passed the FinalAssessment with a score of 75%or better.

STRENGTHS: Instruction of thismaterial was taught with goodexamples, and main points wereclearly communicated tostudents, as demonstrated byhomework and assessmentscores.


Graduates should be able to applyBiased Regression techniques suchas Principal Component Regressionto solve problems of severemulticollinearity.


STRENGTHS: Instruction of thismaterial was taught with goodexamples, and main points wereclearly communicated tostudents, as demonstrated byhomework and assessmentscores.

No changes warranted at thistime

Graduates should be able to finddominant topics/themes in text datausing Nonnegative MatrixFactorization and the Singular ValueDecomposition.


STRENGTH: Inclass exampleusing image compression ofDirector's picture was wellreceived by students and painteda clear picture of this techniquefor dimension reduction. AREAFOR IMPROVEMENT: Can beimproved by teaching thepartitioned matrix approach tomultiplication early so that theactual equation for the singularvalue decomposition is not sodaunting to students.

We have included the partitionedmatrix approach to multiplicationin the Primer so that studentshave actually seen the equationfor the singular valuedecomposition and expanded it inpartitioned form themselves as anexercise.

Objective 19: Project management




Graduates should be able todemonstrate project managementplanning and execution by developingand maintaining a work breakdownstructure identifying all subtasksrequired to plan and complete aproject form start to finish, assignand track individual accountability foreach subtask, schedule and trackwork progress as it occurs, andestimate completion of future work

Practicum Coaching RubricFINDINGS:17 of 17 project teamssubmitted regular biweeklyproject reports capturing livingbacklog, completion timeestimates, individualaccountability and pace/scheduleprojection. Recurring individualfeedback given on each report toteam lead. (Documented onPracticum Teams spreadsheet.)

STRENGTH: ProjectManagement biweekly updateswere accomplished well by allteams. Formal sponsor feedbackwas consistently good regardingplanning and communicationthroughout all projects. AREAFOR IMPROVEMENT: Stricterupdating of scoped objectives isnecessary to this form for nextyear.

We added stricter updating ofscoped objectives to this form fornext year. Made scopingdecisions explicit/transparent atthe time of decision making.

based on previous rate of progress.

Graduates should be able todemonstrate project managementcompetency by successfullyperforming scope feasibility analysison original project scope andsubsequent scopedevelopment/refinement.

Practicum Coaching RubricFINDINGS: Project Midtermpresentation (12/15) tostaff/faculty and then to sponsorused to present scope refinementand solicit strategicguidance/agreement from eachaudience. 17/17 midtermsaccomplished – all achievingobjectives. Very well received bysponsors.

STRENGTHS: This approachseems to work well, as formalfeedback from sponsors isconsistently very positive.AREAS FOR IMPROVEMENT:Enhance objective language formidterm to include more generaltext of "gaining sponsorconfidence by" achieving theexisting objectives of 1)demonstrating descriptiveunderstanding of data, 2)demonstrating understanding ofthe business problem, and 3)detailing plan and solicitingfeedback for accomplishinginferential objectives

Enhanced objective language formidterm to include more generaltext of "gaining sponsorconfidence by" achieving theexisting objectives of 1)demonstrating descriptiveunderstanding of data, 2)demonstrating understanding ofthe business problem, and 3)detailing plan and solicitingfeedback for accomplishinginferential objectives

Objective 20: Teamwork/problem solving/conflict resolution




Graduates should be able todemonstrate professionalism andeffectiveness in teambased settings.

Peer Evaluation RubricFINDINGS:Per peer feedbackweaknesses in these area – alladdressed – not appearing on aregular basis in any student byend of program.

STRENGTH: Individualizedcoaching in response to peerevaluations is a major strength ofthe program. The peerfeedback/coaching process iseffective in producing desiredresults.


Graduates should be able todemonstrate professionalism andeffectiveness in resolving conflictassociated with creative differencesin approaching analyticproblems/projects and with workloaddistribution and management.




Graduates should be able todemonstrate leadership skills inplanning and articulating a vision forhomework team projects, organizingworkload assignments, controlling forperformance variation amongstteammates, and motivatingteammates to succeed.




Graduates should be able todemonstrate followership insupporting and enabling team leadsby contributing effectively to the workplan, vision development andorganization, subordinating/aligningindividual goals with those of theteam, executing assignments, andmotivating teammates and teamleader.




Graduates should be able todemonstrate problem solvingcompetencies by being able toeffectively resolve project issuesfocused on understanding underlyingdatasets to be analyzed,understanding context of theunderlying business problem,articulatingrefiningboundingassumptions and project problemstatements, and then addressingscope feasibility and development.

Peer Evaluation RubricFINDINGS:Per coaching staffand midterm presentation someoccurrences of weaknesses inthese area – all addressed overthe course of experientiallearning– not appearing on aregular basis in any student byend of program. Mid termscoresheets capture coach’sfedback

AREA FOR IMPROVEMENT:Some minor occurrences ofproblem definition issues arearising in midterm presentationswhich could be addressed earlierin practicum coaching meetings.

We have decided to emphasize inthe first coaching staff meetingsrigorous problem definition as ateam deliverable.

Objective 21: Communication skills




Graduates should be able to preparedocuments for use in businesssettings that are direct, concise,professional, easily skimmable, andgrammatically correct; e.g. applyconventions and strategies tobusiness emails, design and writeappropriate memos, create anaudience centered executivesummary, create an audiencecentered, persuasive resume.

Presentation Rubrics, BrodyCommunication RubricFINDINGS: Per coaching staffand midterm presentation someoccurrences of weaknesses inthese area – all addressed overthe course of experientiallearning– not appearing on aregular basis in any student byend of program. Mid termscoresheets capture coach’sfedback

AREA FOR IMPROVEMENT:Some minor occurrences ofproblem definition issues arearising in midterm presentationswhich could be addressed earlierin practicum coaching meetings.

We have decided to emphasize inthe first coaching staff meetingsrigorous problem definition as ateam deliverable.

Graduates should be able to applystrategies for editing and proofreadingthat demonstrate an understanding ofgrammar.

Presentation Rubrics, BrodyCommunication RubricFINDINGS:Per review of finalpracticum deliverables someminor (4 projects of 17)occurrences of weaknesses inthese areas – all addressed overthe course of iterative drafts. Finalscoresheets capture coach’sfeedback

AREA FOR IMPROVEMENT:Professional, coherent, preciselanguage was evident only in aminority of early deliverabledrafts. Additional staff expertiseon site would improve students'technical writing for decisionmakers.

We have decided to increasecoverage in this area with additionof a .75 FTE staff member.Position posted.

Graduates should be able to identify,understand, and use the componentsof an effective presentation.

Presentation Rubrics, BrodyCommunication RubricFINDINGS:Per review of midtermpresentations some (4/17) andfinal presentations (1/17)occurrences of weaknesses inthese areas – all addressed overthe course of iterative drafts. Finalscoresheets capture coach’sfeedback

AREA FOR IMPROVEMENT:Professional, coherent, preciselanguage was evident only in aminority of early deliverabledrafts. Additional staff expertiseon site would improve students'technical writing for decisionmakers.

We have decided to increasecoverage in this area with additionof a .75 FTE staff member.Position posted.

Graduates should be able to assessthe audience for a presentation anduse a variety of modes to present.

Presentation Rubrics, BrodyCommunication RubricFINDINGS:Per review of midtermpresentations some (2/17) andfinal presentations (1/17)occurrences of weaknesses inthese areas – all addressed overthe course of iterative drafts .Final scoresheets capture coach’sfeedback

STRENGTH: Iterative approachto presentations is good practiceand is best way to internalizelearning. AREA FORIMPROVEMENT: The addition ofstaff to model specificcommunication skills (writing andspeaking/legal) would help inproviding better feedback tostudents

We have decided to increasecoverage in technicalcommunication with the additionof a .75 FTE staff member. Planto increase coverage in legal andethical issues of data withaddition of one FTE staff memberwith law expertise. Both positionsposted.

Graduates should be able to developstrategies for preparing andstructuring presentations tocommunicate, motivate, and/orpersuade listeners.

Presentation Rubrics, BrodyCommunication Rubric, Sponsorcomments FINDINGS:Per reviewof midterm presentations some(2/17) and final presentations(1/17) occurrences ofweaknesses in these areas – alladdressed over the course ofiterative drafts . Final scoresheetscapture coach’s feedback



Graduates should be able to usetechniques for connecting with anaudience, including voice projection,pacing, body language, eye contact,gestures, humor, personality andother performance skills.

Presentation Rubrics, BrodyCommunication Rubric, coachand sponsor commentsFINDINGS:BFeedback fromSponsors on final practicumpresentations indicate all students& teams were doing well in thisregard – sponsor feedbackspreadsheet consistentlydocuments presentation skills



Graduates should be able to usemedia and visual aids effectively inpresentations.

Presentation Rubrics, BrodyCommunication RubricFINDINGS:Based on finalpracticum presentations feedbackfrom Sponsors & Coaches, allwere doing well in this regard


We have decided to increasecoverage in technicalcommunication with the additionof a .75 FTE staff member. Planto increase coverage in legal andethical issues of data withaddition of one FTE staff memberwith law expertise. Both positionsposted

Graduates should be able to developstrategies to handle fear of publicspeaking, minimize distractingmannerisms, e.g. verbal pauses, andproject confidence.

Presentation Rubrics, BrodyCommunication RubricFINDINGS: A handful of studentswith such issues –all werecoached and based on feedbackprovided by sponsors & coaches,all did well by time of finalpresentation



Graduates should be able to handleaudience questions.

Presentation Rubrics, BrodyCommunication RubricFINDINGS:Feedback fromSponsors on final practicumpresentations indicate all students& teams were doing well in thisregard – sponsor feedbackspreadsheet consistentlydocuments presentation skills



assessment report viewer to meet enrollment and admissions ... · comparablysized mba programs at...

Documents