jmu assessment progress template interdisciplinary liberal ...1 jmu assessment progress template...
TRANSCRIPT
1
JMUAssessmentProgressTemplate
InterdisciplinaryLiberalStudies(IdLS)-BS
PARTI.Objectives
Descriptionofprocessfordevelopingobjectives:Morethanadecadeago,asmallgroupofadministratorsassembled31minutelydetailedStudentLearningObjectives.ThesewerelargelydefinedbytheVirginiaSOL’sandteacherlicensurecompetenciesineachofthemajorsubjectareas.Whilefairlydetailed,theseobjectiveswerelargelyunassessable.Assuch,arecommendationwasmadeinthe2008-2009IdLSAssessmentProgressTemplatetorevisethelistofIdLSGoalsandObjectives.InSpringof2010,representativesfromIdLSmetwithDr.KestonFulcherfromCARSanddiscussedthedevelopmentofnew,assessable,programGoalsandObjectivesthatfulfilltheIdLSMission.TheMissionofIdLSis:
• Tosupporttheuniversity’smissiontoproduceeducatedandenlightenedcitizens.• Tohelpstudentsembracewisdom,inspirelearning,andenhanceliving.• TomeetVirginiateachercompetenciesbyprovidingbreadthandintegrationacrossthecontent
areasofEnglishandlanguagearts,history,socialsciences,mathematics,naturalsciences,andtechnology.
• ToworkcollaborativelywiththeEducationUnittoreachitsgoalsasarticulatedinitsConceptualFramework,particularlyastheyrelatetodevelopingadeepunderstandingofcontent.
Startingwiththe2009-2010APTthefollowinglearningobjectivesserveasAPTProgramGoalsandObjectivesfortheIdLSprogram*.
Goals Objectives MeasuresandRationale
IdLSstudentscompletingGeneralEducationwilldemonstrateknowledgecentraltotheuniversity’smission,andrelevanttotheVirginiateachercompetencies.
IdLSstudentswill,asagroup,matchotherJMUstudentsonGeneralEducationlearningoutcomes,specificallyintechnology,informationliteracy,scientificreasoning,quantitativereasoning,andtheglobalandAmericanexperiences.
IdLSvsnon-IdLSdatafromClusters1(MREST*test),3(ScientificReasoning&QuantitativeReasoningsub-scales),and4(Global&AmericanExperiencetests).
GeneralEducationisthebaseuponwhichJMUaimstofulfillitsmissiontoproduceeducatedandenlightenedcitizens,andthespecificcontentareasmeasuredareteachercompetenciesrequiredbytheVirginiaDepartmentofEducation.
IdLSseniorswilldemonstratecontent-areaproficiencyonteacherlicensure
ForallIdLS-relatededucationprogramshavingPRAXISIIcontent-arealicensureexams(i.e.,ElEdandMIED),eachprogram
PRAXISIIscoresandpassrates,reportedbyeducationprogram.
Content-areaproficiencyismeasuredbyPRAXISIItests,andan80%passrateon
2
exams. willhaveatleastan80%passrateonthoseexams.
PRAXISIIisaminimumpassraterequiredbyNCATEforaccreditation.
ForELEDandIECE,requiredcontent-areaknowledgeiscoveredbytheIdLScore,takenbyallELEDandIECEstudents.ForMIEDstudents,requiredcontent-areaiscoveredbytheIdLSMIEDcoreandupper-levelconcentrations.SPEDstudentsdonothaveacontent-relatedexam.
IdLSgraduateswillapplycontent-areaproficiencyinpK-8classroomsettings.
ForcurrentJMUMATstudentswhocompletedtheIdLSmajor,morethan80%willgetconfirmationthattheyappropriatelyappliedcontent-knowledgeduringtheirstudentteachingassignments.
ST-9data(itemA2,“Identifieskeyprinciplesandconceptsofsubjectmatter”)completedbystudentteachersupervisorsandJMUinstructors.
Whilethe80%passrateisnotrequiredbyanyaccreditingbody,thiscontent-relatedpassratemirrorsthePRAXISIIpassraterequiredbyNCATEandisthereforeanappropriateminimumexpectationforcontentproficiency.
Table1.IdLSGoals,Objectives,andMeasures
*Asdiscussedinthefollowingtext,startingwiththe2013-2014reportingyear,theISST(InformationSeekingSkillsTest)testhasbeenreplacedbytheMREST(MadisonResearchEssentialSkillsTest).PartII.Course/LearningExperiencesVirginiarequiresallofitsteachercandidatestobepreparedtoteachthematerialinalloftheSOLfortheareaoflicensure,thereforetheIdLSprogramgoalsandobjectivesmustmeshwiththestateandfederalrequirementsforteachereducation.In2005-06,IdLSfacultyconductedthefollowingalignmentsofourcurriculum. Math/Science Humanities/SocialScience Core Concentration Core ConcentrationVA–SOLElementary X X VALicensureStandards–ElementaryEducation
MathOnly MathOnly X X
VALicensureStandards–MiddleEducation
X X X X
SPAStandards ScienceOnly ScienceOnly Table2.AlignmentsconductedforIdLScurriculumandaccreditation/licensurestandards,2005-06.
Resultsofthesealignmentstudiesrevealedthatourcorecurriculuminbothmath/scienceandhumanities/socialsciencesincludesnearlyalloftheessentialcomponentsforteacherlicensure.Afew
3
specificsubjectareasinsciencehavelittleornocoverage(weather,plants,soil,technologyforexample)andinlanguageartsstudentsareexposedtooneoratmosttwoofthe4literatureareas(American,British,World,Ethnic)butoverallthecorecurriculumprovidesanexcellentfoundationinall4subjectareas.Theconcentrationcurriculumwasevaluatedintwoways.First,transcriptsofallrecentMiddleEducationgraduateswereanalyzed.Sincestudentshavemanychoicesintheirconcentrationcoursework,itwasfeltthatdirectevaluationoftranscriptswouldgivethebestinformationofwhatisactuallycoveredinstudents’programs.ThesedataarefoundintheAppendicesofthe2009-2010report.Second,thecoursesthemselveswereanalyzedfortheSOLorlicensureareasthattheinstructorscoverinthecourse.ThesedataarealsofoundintheAppendicesofthe2009-2010report.TranscriptevaluationsshowedthatmostoftheMIEDhumanities/socialsciencesstudentschoosecoursesthatcoverlessthanhalfoftherequiredlicensurecompetencies.Particularweaknesseswereincivics/economicsandworldhistory.Worldhistoryiscoveredextensivelyinthecore,butcivic/economicscoverageappearsweakinbothcoreandconcentration.Math/scienceMIEDconcentrators’transcriptswerenotevaluatedinthesameway,becausethesciencecomponentofthisconcentrationhaschangedsignificantlyinthepastseveralyears.Thisevaluationshowedthatstudentsarechoosingcourseswhichfallintooneortwosciencedisciplines(astheoldguidelinesrecommended).Thenewconcentrationguidelinesaremorerestrictiveofcourseselectionsandrequireabroaderchoiceofdisciplineareas.Futureevaluationsneedtobedonetodetermineifcoverageisimproved.Theindividualalignmentsarefoundintheappendicesassociatedwiththe2005-06report.
4
PartIII.Evaluation/AssessmentMethodsIdLSassessmentisverycomplex.Evaluatingstudentswithtwodistinctupperdivisionconcentrations,fortheirmasteryofknowledge,skills/attitudesineachof4subjectareasposesachallenge.Thankfullyseveralfacultyanddepartmentshavebeenextraordinarilyhelpfulinassemblingdataforourevaluation.CARSstaffhavedoneanalysesofGeneralEducationdata(Clusters1,3,and4)thatidentifyIdLSstudentsandcalculatetheirscoresseparately.TheEducationalSupportCenterintheCoEhasprovideddatabasequeriesandprovidedstudentinformationregardingPRAXISIItestresultsandST-9results.(SeeTable1foradescriptionofeachoftheinstrumentsused(ClustersinGenED,PRAXISII,andST-9)andwhyIdLSchosetousethem).Table3belowindicatesthecurrentstatusofassessmentsforcandidates’knowledgeandskills/attitudesineachofthefourcoresubjectareas.
SubjectArea InstrumentsUsedtoEvaluateCandidates’Knowledge Skills/Attitudes
Science Cluster3,PRAXISII ST-9Math Cluster3,PRAXISII ST-9LanguageArts Cluster1,PRAXISII ST-9SocialStudies Cluster4,PRAXISII ST-9Table3.IdLSassessmentmethodsgroupedbysubjectareaversusknowledgeorskill/attitude
GeneralEducationInstrumentsTheCorecomponentoftheIdLScurriculumincludesallGenEdrequirementsandallowsustocompareperformanceofIdLSstudentstonon-IdLSstudentsusingthefollowingGeneralEducationassessmenttests:MadisonResearchEssentialSkillsTest(MREST),NaturalWorld(NAW)quantitativereasoning,NaturalWorldscientificreasoning(NAW),GlobalExperience(GLEX),andAmericanExperience(AMEX).Aswasthecaseforacademicyears2008-2009,2009-2010,2010-2011,2011-2012,2012-2013,2013-2014,and2014-2015,dataforthisyear(2015-2016)wereevaluatedontheperformanceofIdLSstudentsfortheMREST,theNaturalWorldQR&SR,theGlobalExperience,andAmericanExperiencetests.Generaldescription,datacollectioninformation,anddesiredresultsareprovidedforeachofthesegeneraleducationtestsbelow.MadisonResearchEssentialsTest(MREST)(oldInformationSeekingSkillsTest(ISST))Startingwiththe2013-2014year’sAPTtheMadisonResearchEssentialsTest(MREST)hasbeenusedtoassessthestudentsabilityto:(http://www.jmu.edu/gened/infor_lit_general.shtml)
1. Recognizethatinformationisavailableinavarietyofformsincluding,butnotlimitedto,text,images,andvisualmedia.
2. Determinewheninformationisneededandfinditefficientlyusingavarietyofreferencesources.
3. Evaluatethequalityoftheinformation.4. Useinformationeffectivelyforapurpose.5. Employappropriatetechnologiestocreateaninformation-basedproduct.
5
6. Useinformationethicallyandlegally.Priorto2013-2014,theexamusedforthisassessmentwastheInformationSeekingSkillsTest(ISST).AccordingtoDeMars,Cameron,andErwin(2003),“theISSTisaweb-basedtestof53multiple-choiceitems.Fourcontentareas(BasicReference,DatabaseSearching,InternetSkills,Ethics)arecrossedwithtwoprocessareas(Knowledge,Application).Applicationquestionsrequirestudentstoapplyknowledgebyfindinganswersincatalogsanddatabasesandbyevaluatingwebsites.Proctorsadministerthetestinacomputerlab”.(http://muse.jhu.edu/journals/journal_of_general_education/v052/52.4demars.html)Becausefirst-yearstudentsmustpassthetestbeforeenrollinginsophomorecourses,studentstypicallygiveagoodeffortonthistest.PracticallyallIdLSstudentstakethistest(i.e.,acensus).TheexactnumberofIdLSstudentswhotookthetestisprovidedintheresultssection.Reliabilityanalysesoverthepastseveralyears(viaitemresponsetheory)revealthatthereliabilityfortheentiretestisinthelowtomid.70’s,areasonablelevelformakinggroupdecisionsinhighereducation.LibrariansdevelopedthistestandstudiesbyCARShaveindicatedthatstudentswhohavehadmoreexposuretoinformationliteracycurriculum(e.g.,inclassworkorpracticewithwebmodules)performbetteronthetest.Thesefactorsprovidevalidityevidencethatthescoresonthistestrepresentinformationliteracy.ThedesiredoutcomeisthatIdLSstudentsexhibitthesamedegreeofcompetenceasnon-IdLSstudentsontheMREST.NaturalWorldTestVersion9,ScientificReasoningandQuantitativeReasoningScoresTheNW-9testconsistsof66items,allofwhichcontributetothescientificreasoning(SR)score.Twenty-sixofthoseitemsalsocontributetoquantitativereasoningandaretotaledfora“QR”subscore.Thistestisdeliveredviapaperandpencilandcomputer-basedversions,bothinthecontextofAssessmentDay.Itistypicalthatapproximatelyonequarterofenteringfreshmenarerandomlyassigned(viathelasttwodigitsofastudent’sID)totaketheNAW-9duringafallAssessmentDay.Unfortunately,theSpring2015NAW-9hadanextremelysmallsamplesizeduetosomeaflawedexperimentaldatadesignswhichresultedinonly4IDLSstudentsbeingmatchedfrompretesttoposttest.Thisunfortunatelymakesstatisticaltestsinappropriate.Asaresultthisyear’sreportwillnotpresentNAW-9testresultsforthisyearbutwillinsteadpresentthepreviousyear’stestresultswhereastatisticallysignificantnumberofIdLSstudentstooktheNAW-9asenteringfreshmenandretookthetestinthespringof2014.PrioryearsstudentsselfreportthattheyaremotivatedtotakethisexamandgiveareasonableeffortontheNAW-9.ThereliabilityoftheSRandQRscoresaretypicallyinthe.70sand.60s(Cronbach’salphas)respectively.Thislevelofprecisionisrespectableforhighereducationtestsforgroup-leveldecisions.Thetestwasdesignedbyfacultycontentexpertsandthesescoresrelatetobothcourseexposureandcoursegradesinscienceandmath.Thesefactorscontributetovalidityevidencethatthescoresdoindeedreflectquantitativeandscientificreasoning.Intermsofdesiredresults,theIdLSprogramwouldlikeIdLSsophomores(post-test)toscorethesameasotherJMUstudents.Additionally,theIdLSprogramwouldlikeIdLSstudentstomakesimilargainsfrompre-testtopost-testasnon-IDLSstudents.ThesecriteriafordesiredresultsarebaseduponpreviousdataprovidedbyCARS.Global(GLEX)andAmericanExperience(AMEX)TestsTheGLEXinstrumentconsistsof31multiplechoiceitems,AMEXconsistsof81multiplechoiceitems.ThetestsareadministeredtoincomingFreshmenduringtheAugustassessmentday,andtostudents
6
with45-70credithoursduringtheSpringassessmentday.Testsweredevelopedbycontentareafaculty.Scoresonbothtestsarestandardizedtoameanof500andstandarddeviationof100,setsotheymatchthemeansofthenorminggroupsforthetests(freshmenin2000or2001).ThereliabilityoftheAMEXtestisconsistentlyintherangeof0.87,theGLEXistypicallyintherangeor0.75(Cronbach’salpha).Thesereliabilitiesaresufficienttomakegroup-leveldecisionsbasedonaggregatedscores.PRAXISIIAllteacherlicensurecandidatesmustpasstherelevantPRAXISIIexam(s)inordertobelicensed.TheseexamsaredevelopedatEducationalTestingServices(ETS)inconsultationwithteachingexpertsacrossthenation.Inessence,thetestsaredesignedtocorresponddirectlywithteachinglicensureobjectives.ETSprovidesreliabilityandvalidityevidenceforthistest:http://www.ets.org/s/praxis/pdf/validity.pdf.Thereliabilitiesofthese5testsrangefrom0.88to0.90nationally.Becauseapassingscoreisrequiredforlicensure,studentsareassumedtoprovideagoodeffortonthistest.Forthepastseveralyears,scorereportsandinstitutionalsummariesofJMUdatahavebeenavailablefromETS.FortheElementaryEducation(ElEd)ContentKnowledgetest,scoresareprovidedforeachofthe4subjectareasubscales.Startingwiththisyears’APTreport,IdLSstudentsarerequiredbytheVirginiaDepartmentofEducation(VDOE)topasseachofthefoursubjectareas(Science,Math,LanguageArts,andSocialStudies)tests.InallpriorreportsIdLSstudentspassed(orfailed)thePRAXISIIexambasedonanaggregationofscoresforallfoursubjectareas(Science,Math,LanguageArts,andSocialStudies)whereeachareacontributedequally(25%)tothetotalscore.ElEdPRAXISIIdatareportedforthisyears’APTissomewhatconfusingasaresultoftheentireexaminebeingrewrittenandscoresbeingrecalibrated.Foreachofthefoursubjectareastherearetwoversionsoftheexamthatwillbereported.Thebreakdownofcontentontheexamsisasfollows.From9/14/14to6/30/15ElEdReadingandLanguageArtsContentCategories(Test5032)
ApproximatePercentageofExamination
I.Reading 49%II.Language,Writing,andCommunication 51%(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdReadingandLanguageArtsContentCategories(Test5002)
ApproximatePercentageofExamination
I.ReadingII.Writing,Speaking,andListening
47%53%
(source:https://www.ets.org/s/praxis/pdf/5001.pdf)From9/14/14to6/30/15ElEdMathematicsContentCategories(Test5033)
ApproximatePercentageofExamination
I.Number,Operations,andAlgebraicThinking 65%
7
II.Geometry,Measurement,Data,andInterpretation 35%(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdMathematicsContentCategories(Test5003)
ApproximatePercentageofExamination
I.NumbersandOperationsII.AlgebraicThinking
40%30%
III.GeometryandMeasurement,Data,Statistics,andProbability
30%
(source:https://www.ets.org/s/praxis/pdf/5001.pdf)From9/14/14to6/30/15ElEdSocialStudiesContentCategories(Test5034)
ApproximatePercentageofExamination
I.UnitedStatesHistory,Government,andCitizenship 45%II.Geography,Anthropology,andSociologyIII.WorldHistoryandEconomics
30%25%
(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdSocialStudiesContentCategories(Test5004)
ApproximatePercentageofExamination
I.UnitedStatesHistory,Government,andCitizenshipII.Geography,Anthropology,andSociology
40%30%
III.WorldHistoryandEconomics 30%(source:https://www.ets.org/s/praxis/pdf/5001.pdf)From9/14/14to6/30/15ElEdScienceContentCategories(Test5035)
ApproximatePercentageofExamination
I.EarthScience 32%II.LifeScienceIII.PhysicalScience
34%34%
(source:https://www.ets.org/s/praxis/pdf/5031.pdf)After7/1/15ElEdScienceContentCategories(Test5005)
ApproximatePercentageofExamination
I.EarthScienceII.LifeScience
32%33%
III.PhysicalScience 34%(source:https://www.ets.org/s/praxis/pdf/5001.pdf)TableSet4A.ContentareacoverageandexambreakdownforthefourElementaryEducationPraxisIIcontentexams.
8
Eachofthe4MiddleSchoolsubjectareatestscontainsseveraldiscipline-relatedscales(seebelow).ETSpublishesthelistofcontentknowledgethatisusedtodevelopthetest;thisappearstomatchtheIdLSlearningobjectivesfairlywell.ETSrecommendsthatPRAXIScontentbealignedwithcurriculumandlearningoutcomesbeforeusingittomakedecisionsaboutprograms.Thebreakdownofcontentontheexamsisasfollows(NOTE:VDOErequiredstudentstakingtheMathematicsandLanguageArtscontentexamstotakedifferentexamsafterJan.1,2014,thereforebreakdownsareprovidedforbothoftheseexams):PriortoDec31,2013MiddleSchoolMathematicsContentCategories
ApproximatePercentageofExamination
I.ArithmeticandBasicAlgebra 20%II.GeometryandMeasurement 17%III.FunctionsandTheirGraphs 13%IV.Data,Probability,andStatisticalConcepts;DiscreteMathematics
17%V.Problem-SolvingExercises 33%ProcessCategories(DistributedAcrossContentCategories)MathematicalProblemSolving,MathematicalReasoningandProof,MathematicalConnections,MathematicalRepresentation,UseofTechnology(source:http://www.ets.org/s/praxis/pdf/0069.pdf)AfterJan1,2014MiddleSchoolMathematicsContentCategories
ApproximatePercentageofExamination
I.ArithmeticandAlgebra 62%II.GeometryandData 38%(source:http://www.ets.org/s/praxis/pdf/5169.pdf)PriortoDec31,2013MiddleSchoolLanguageArtsContentCategories ApproximatePercentageofExaminationI.ReadingandLiteratureStudy 37%II.LanguageStudy 13%III.CompositionandRhetoric 25%IV.ShortEssays
1.TextualInterpretation,2.TeachingReading/Writing
25%(source:http://www.ets.org/s/praxis/pdf/5049.pdf)
AfterJan1,2014MiddleSchoolLanguageArtsContentCategories ApproximatePercentageofExaminationI.Reading 46%II.LanguageUseandVocabulary 11%
9
III.Writing,Speaking,andListening 18%IV.EnglishLanguageArtsInstruction 25%
(source:http://www.ets.org/s/praxis/pdf/5049.pdf)
MiddleSchoolScienceContentCategories ApproximatePercentageofTotalScoreI.ScientificMethodology,Techniques,andHistory 8%II.BasicPrinciples 11%III.PhysicalSciences 18%IV.LifeSciences 15%V.Earth/SpaceSciences 15%VI.Science,Technology,andSociety 8%VII.ShortContentEssays:
1.PhysicalSciences,2.LifeSciences,3.Earth/SpaceSciences
25%(source:http://www.ets.org/s/praxis/pdf/0349.pdf)
MiddleSchoolSocialStudiesContentCategories ApproximatePercentageofExaminationI.UnitedStatesHistory 19%II.WorldHistory 15%III.Government/Civics 14%IV.Geography 14%V.Economics 13%VI.ShortContentEssays 25%(source:http://www.ets.org/s/praxis/pdf/0089.pdf)TableSet4B.ContentareacoverageandexambreakdownforfourMiddleSchoolPraxisIIcontentexams.ST-9ST-9ispartofthe“AssessmentofStudentTeaching”conductedbytheCOEatJMU.Thisform(seeAppendix1),titled“PROFILEOFSTUDENTTEACHINGPERFORMANCE”isfilledoutbythecooperatingteacheranduniversitysupervisorwhiletheIdLSstudentisStudentTeaching.BoxA2ofthisformpertainstotheabilityoftheSTUDENTTEACHERtoIDENTIFYKEYPRINCIPLESANDCONCEPTSOFSUBJECTMATTER.Ascoreof:
• 3.0meansthatthestudentteacherexplicitlyreferencesANDclearlyalignsappropriatecontentstandardswithplannedactivitiesandassessments,
• 2.0meansthatthestudentteacherexplicitlyreferencesappropriatecontentstandardsindailyplans.
• 1.0meansthatthestudentteacherinaccuratelyandvaguelyreferencesORdoesnotreferenceappropriatecontentstandards.
10
ThemostrecentdatathatisavailablefromtheCOEisforthe2013-2014AcademicYear,andiswhatwillbepresentedhere.
11
PARTIV.ObjectiveAccomplishments/ResultsGENERALEDUCATIONCluster3NW-9TestResults:AsstatedinPartIIItheSpring2015NAW-9hadanextremelysmallsamplesizeduetosomeexperimentsmissingdatadesignswhichresultedinonly4IDLSstudentsbeingmatchedfrompretesttoposttest.Thisunfortunatelymakesstatisticaltestsinappropriate.ThereforethefollowingpresentationofresultsdonotincludedatafortheSpring2015IdLScohort.TheNaturalWorld(NW-9)instrumentmeasuresgeneralscientificreasoningandanalysisskills,independentofspecificcontent.Assuch,itisagoodtestofstudents’overallscienceabilityorskill,butnotoftheirspecificsubjectareaknowledge.AsshowninthelastcolumnofTable4A,IdLSstudents(n=21),onaverageanswered73.88%(SD=12.40)ofNW-9itemscorrectly;whereasnon-IdLSstudents(n-384)answeredonaverage72.81%(SD=13.53)correctly.Thesedifferences(forSpring2016)failedtobestatisticallysignificantt(403)=0.356,p=.700,d=.08.However,thedifferencesinSpring2014werestatisticallysignificantt(973)=3.12,p=.002,d=.20.Stateddifferently,theaverageIdLSNW-9scoreisapproximately.20standarddeviationsbelownon-IdLSstudents.Note:Table4Apresentsdataforthelast4NW-9testresults.Startingwiththemostrecentreportingperiod(Spring2014),CARSispresentingdataas“Mean%”correct.Inpreviousyears,dataispresentedas“Meancorrectresponses”.
NW-9DescriptiveStatisticsforTotalScore Spring2012 Spring2013 Spring2014 Spring2016
MeanCorrect
SD N MeanCorrect
SD N Mean%Correct
SD N Mean%Correct
SD N
Non-IdLS 49.21 7.62 943 49.08 7.63 1231 74.76 11.84 919 73.88 12.40 384
IdLS 48.03 6.73 66 46.49 7.08 74 69.59 11.61 56 72.81 13.53 21
Table4A.ComparisonofNW-9Scoresofnon-IdLSandIdLSstudentsSpring2014andthethreepreviousyears.
AsimpliedbydatashowninthelastcolumnofTable4B,similarresultsareobtainedwithQuantitativeReasoning(QR)scorest(404)=.049,p=.001,d=.01.
Note:Table4Bpresentsdataforthelast4QRtestresults.Startingwiththemostrecentreportingperiod(Spring2014),CARSispresentingdataas“Mean%”correct.Inpreviousyears,dataispresentedas“Meancorrectresponses”.
NW-9DescriptiveStatisticsforQuantitativeReasoning(QR) Spring2012 Spring2013 Spring2014 Spring2016
Mean SD N Mean SD N Mean% SD N Mean% SD N
12
Correct Correct Correct CorrectNon-IdLS
18.55 3.73 943 18.49 3.79 1231
71.45 14.53 919 69.59 0.78 384
IdLS 17.92 3.43 66 17.31 3.72 74 64.22 14.67 56 69.43 3.08 21
Table4B.ComparisonofQRScoresofnon-IdLSandIdLSstudentsforSpring2014andthethreepreviousyears.
InordertodetermineifthesedifferenceswereafunctionCluster3coursework,thisanalysiswasreplicatedwhenholdingCluster3courseworkrequirementsconstant.Table4CprovidesdescriptivestatisticsforperformanceontheNW-9andQRonlyforstudentswhocompletedtheirCluster3requirements.
IdLSstudentswhocompletedrelevantcourseworkonaverageanswered70.48%(SD=14.04)oftheNW-9correctly;whereassimilarnon-IdLSstudentsanswered77.01%(SD=11.10)oftheNW-9correctly.Thesedifferenceswerestatisticallysignificantt(295)=2.65,p=.009,d=.33.ThesevaluesindicatethattheaverageIdLSstudent,whocompletedtheircoursework,isapproximately1/3ofastandarddeviationbelowtheaveragenon-IDLSstudentontheNW-9.Onceagain,similarresultswereobtainedforQRt(295)=2.38,p=.02,d=.29.
Note:DatapresentedinTables4C-4Earenotavailableforpreviousyears.
Table4C.Cluster3MeanScoresandStandardDeviationsforStudentswhoCompletedRelevantCourseworkforSpring2014
Ofthe56IdLSstudentswhocompletedtheNW-9inFebruary2014,therewere23whocompletedtheirCluster3requirements.Ofthese,18completedapre-testasenteringfreshmen.Descriptivestatisticsforthesestudents,aswellassimilarnon-IdLSstudents(n=192)arereportedinTable4DfortheNW-9.BothIdLSandnon-IdLSstudentsincreasedinNW-9atpost-testF(1,208)=15.33,p<.001.Thereis
Spring2014 Spring2015andSpring2016
NW-9 QR
Mean%
Correct
SD N Mean%Correct
SD Mean%Correct
SD N Mean%Correct
SD
Non-IdLSstudents
77.01 11.10 274
73.72 14.08
NoData NoData
NoData
NoData NoData
IdLSstudents
70.48 14.04 23
66.38 15.65
NoData NoData
NoData
NoData NoData
13
currentlyinsufficientevidencetoindicatethatthisincreaseisafunctionofwhetherthestudentisanIdLSornon-IdLSstudentF(1,208)=0.19,p=.67.Stateddifferently,itappearsthatbothIdLSandnon-IdLSwhocompletedrelevantCluster3courseworkshowsimilargainsatpost-test.
Pretest Posttest Difference
Non-IdLSstudents(N=192) 48.33(6.16) 51.31(6.89) 2.98
IdLSstudents(N=18) 44.83(6.88) 47.22(7.35) 2.39
Table4D.NW-9Pre-PostComparisonsforStudentswhoCompletedrelevantCluster3CourseworkforSpring2014(Note.Valuesreportedarebasedupon66totalitems)
AsimilaranalysiswascompletedforQRscores(seeTable4E).Onceagain,IdLSstudentsandnon-IdLSstudentsshowedsimilar,thoughstatisticallyinsignificant,gainsatpost-testF(1,208)=0.48,p=.49.
Pretest Posttest Difference
Non-IdLSstudents(N=192) 18.50(3.40) 19.37(3.40) 0.87
IdLSstudents(N=18) 16.94(3.87) 17.28(3.41) 0.34
Table4E.QRPre-PostComparisonsforStudentswhoCompletedrelevantCluster3CourseworkforSpring2014(Note.Valuesreportedarebasedupon26totalitems)
Cluster4GlobalExperienceandAmericanExperienceTests
TheseinstrumentsareusedtoassessperformanceinClusterFourofGeneralEducation.Ofthisyear’sIDLSstudents,27tooktheAmericanExperiencetestandtheGlobalExperiencetestinFebruary2016.MeanstandardizedscoresareshowninTable5A.Thestandardizedscoresweredefinedtohaveameanof500andastandarddeviationof100inthenorminggroupofenteringfreshmen,soa10-pointdifferenceisabout.10standarddeviationunits(comparabletoCohen’sd,butwithaconstantdenominatorthatdoesn’tchangefromyeartoyearorgrouptogroup).
Spring2015 Spring2016
AmericanExperience
GlobalExperience
AmericanExperience
GlobalExperience
N Mean(sd) N Mean(sd) N Mean(sd) N Mean(sd)
IdLSstudents 45 526.9(89.8)42
572.7(94.8)
27 520.6(100.0)27 517.4(109.4)
Non-IdLS 864 529.1 897583.9
604 520.5(115.4) 463 578.5(104.7)
14
students (118.0) (109.7)
Table5A.StandardizedscoresontheAMEXandGLEXforIdLSstudentsandNon-IdLSstudents(StandardDeviation).
AhigherproportionofIDLSstudentscompletedtheAmericanrequirements(85%ofIDLSvs.69%ofnon-IDLS).InGlobal,aboutthesameproportionsofIDLS(70%)andnon-IDLS(68%)studentscompletedtherequirement.Amorereasonablecomparisoncanbemadebetweenstudentswhohadcompletedtherelevantrequirement.ThesevaluesareshowninTable5B.
Spring2015 Spring2016
AmericanExperience
GlobalExperience
AmericanExperience
GlobalExperience
N Mean(sd) N Mean(sd) N Mean(sd) N Mean(sd)
IdLSstudents 40 532.7(90.3)24
580.6(98.9)
23 532.3(99.8)19
524.9(102.3)
Non-IdLSstudents
608 543.7(117.7) 503
596.3(108.4)
417 535.6(114.2)317
586.0(102.3)
Table5B.AMEXandGLEXscoresforstudentswhohadcompletedtheAmericanorGlobalExperiencerequirement.
FromTable5B,thenon-IDLSstudentsscored3points(approximately0.03standarddeviationunits)higher.Thisdifferencewasnotstatisticallysignificant(t438=0.14,p=.893),whichmeansthatadifferenceofthismagnitudecouldbeduetochancevariation.Theplausiblerangeforthedifferencerangedfromabout-44.5to51.1,onthestandardscale.InGlobalExperience,thenon-IDLSstudentsscored0.60standarddeviationunitshigher.Althoughthisseemslikealargedifference,duetothesmallsamplesizewehavelittleconfidenceintheestimateofthedifference;theplausiblerangeofthedifferenceisabout13.5to108.6onthestandardscale.Thus,althoughthedifferencebetweentheIDLSstudentscoresandthenon-IDLSstudentscoreswasstatisticallysignificant(t334=2.53,p=.012),thedifferencecouldbequitesmallorverylarge.
Ofthe27IDLSstudentstestedontheAmericanExperience,18hadscoresfromthefall1.5or2.5yearsbefore.Similarly,21oftheIDLSstudentshadearlierscoresontheGlobalExperiencetest.15ofthesestudentshadcompletedoneoftheAmericanExperiencecoursesatJMU,and15hadcompletedoneormoreoftheGlobalExperiencecourses(another5hadtransferorAPcredit).Fifteenstudentsisaverysmallnumber,sounlikepreviousyearstheCARSprogramdidnofurtheranalysesofpretopostdifferences.
Tables5Cand5Dshowpre-andpost-testdifferenceanalysisfortheprevioustwoyears.For2015(themostrecentyearwherethisdataisavailable)ofthe45IDLSstudentstestedontheAmericanExperience,34hadscoresfromthefall1.5or2.5yearsbefore.Similarly,34oftheIDLSstudentshadearlierscoresontheGlobalExperiencetest.26ofthesestudentshadcompletedoneoftheAmerican
15
ExperiencecoursesatJMU(another5hadtransferorAPcredit),and21hadcompletedoneormoreoftheGlobalExperiencecourses(another4hadtransferorAPcredit).Thespringtestservedasaposttestforthesestudents.UnlikeTables2and3,thepre-postcomparisondoesnotincludestudentswhocompletedtherequirementbyAPortransfercreditbeforethepretestanddidnottakeanadditionalcourseatJMU(thesestudentswouldnotbeexpectedtoincreasetheirscores).MeanscoresareshowninTables5Cand5D.
AmericanExperiencePre-PostComparisons
2014Data 2015Data
N Pretest(sd)
Posttest(sd)
Difference N Pretest(sd)
Posttest(sd)
Difference
IdLSstudents 24 468.0
(107.7)516.4(103.0) 48.4 26 471.7
(90.3)527.0(92.6) 55.3
Non-IdLSstudents 273 491.9
(99.3)522.5(102.5) 30.6 307 495.1
(103.6)532.1(101.1) 37.0
Table5C.Pre-andPost-testcomparisonsforAmericanExperience(StandardDeviation).
GlobalExperiencePre-PostComparisons
2014Data 2015Data
N Pretest(sd)
Posttest(sd)
Difference N Pretest(sd)
Posttest(sd)
Difference
IdLSstudents 23 522.8(137.2)
587.9(123.6) 65.1 21 520.8
(86.0)584.3(97.1) 63.5
Non-IdLSstudents 560 543.3
(109.3)586.3(117.5) 43.0 387 542.2
(106.1)604.8(103.8) 62.6
Table5D.Pre-andPost-testcomparisonsforGlobalExperience(StandardDeviation).
OntheAmericanExperiencetest,theinteractionbetweenIdLS/notIdLSandpre/posttestwasnotstatisticallysignificantthisyear(F1,331=1.25,p=.264).Inotherwords,thenon-IdLSincreasewasnotsignificantlydifferentfromtheIdLSincrease.Withsuchasmallsample,themeandifferencesareunstable.
OntheGlobalExperiencetest,therewasnotasignificantinteractionbetweenIdLS/non-IdLSandpre/posttest(F1,406=0.00,p=.965).Inotherwords,thenon-IdLSincreasewasnotsignificantlydifferentfromtheIdLSincrease,asonewouldexpectgiventhatthedifferenceswerenearlyequal.
16
Cluster1(MRESTtest)
Startingin2014,theCluster1examiscalledtheMadisonResearchEssentialSkillsTest(MREST).TherearetwoformsoftheMREST,soscoresareagainreportedonlyonthestandardizedscale.Scoresrangefrom100-200.TheMRESTcontainsitemsthatwereadministeredwiththeISSTin2012-2013,sothescalewassetwithameanof150andstandarddeviationof15forthe2012-2013students.Thepassingscorewassetbyafacultycommitteeat148.Scoresof166orgreaterreceiveanAdvancedtranscriptnotation.Studentsmayrepeatthetestanunlimitednumberoftimes,andtutorialsareavailable.Nearlyallstudentspassbytheendofthe1styear(thosewhodonotprobablydidnotbotherrepeatingthetestiftheydidnotintendtoremainatJMU).Thisyear(2016Data)IdLSandnon-IdLSstudentsperformedapproximatelythesameontheMRESTexam(seeTable6A).100%ofIdLSstudentspassand99%ofnon-IdLSstudentspass,and22%ofIdLSstudentspassadvancewhile21%onnon-IdLSstudentspassadvance.PercentPassingMREST(ISST)(ofthosewhoattemptedthetestatleastonce)
2015Data(MREST) 2016Data(MREST)
N
#Pass
%Pass
#Advanced
%Advanced
N#
Pass%
Pass#
Advanced%
Advanced
IdLS 186 186 100% 49 26% 184 184 100% 41 22%
NonIdLS
3997 3949 98.8% 964 24% 4016 3980 99% 836 21%
Table6A.PercentPassingeithertheISSTortheMREST(ofthosewhoattemptedthetestatleastonce)
IdLSstudentsattemptedthetestanaverageof1.83times,almostthesameasthenon-IDLSstudentswith1.86attemptsonaverage.Scoresfromthe1stattemptandfinalattemptareinthetablebelow(seeTable6B).Formanystudents,the1stattemptwasalsothefinalattempt;onlythosewhodidnotpassrepeatedthetest.Thus,scoresincreaseandthestandarddeviationdecreasesforthefinalattempt.
2015Data(MREST) 2016Data(MREST)
1stAttempt FinalAttempt 1stAttempt FinalAttempt
Mean SD Mean SD Mean SD Mean SD
IdLS 153.1 13.6 159.9 10.2 149.2 13.1 158.9 8.52
Non-IdLS 152.8 14.6 159.6 10.1 150.4 14.3 158.8 9.43
Table6B.Meanscoresfor1standfinalattemptsforeithertheISSTortheMRESTforIdLSstudentsandothers.
17
IdLSstudentsscorednearlythesameasnon-IdLSstudents.Thedifferencebetweennon-IdLSandIdLSstudentscoreswasvirtuallyzeroandnotstatisticallysignificant[firstattempt:t4198=1.13,p=.259,finalattempt:t4198=-0.10,p=.917].
18
PRAXISIIElementaryEducation(ElEd)ContentKnowledgeAsofthewritingofthisreport,theElEdContentKnowledgetestsarehigh-stakesassessmentsinwhichstudentsmustpassallfoursubjectareaexams(Reading&LanguageArts,Mathematics,SocialStudies,andScience)inordertoreceivelicensuretoteachinVirginia.ThetestgenerallymatchestheIdLSprogramcorecurriculumsincethisiscontentthatallelementaryteachersmustteach.
ForthisreportingperiodtherearetwoexamsthatwereadministeredbyPRAXISandbothoftheseexamsaredifferentthanthePRAXISIIteststhatwerereportedinpreviousAPT’sforIdLS.Theresultofthisisthatitisdifficulttomakedirectcomparisonsbetweenthisandpreviousyears’data,howeverforcompletenessdataforthreepreviousyearsofPRAXISIItestresults(TablesIandJ)anddiscussionofdataareincludedattheendofthissectionforcomparison.
-ElEdReading&LanguageArts
Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forexam#5032thepassingscoreis165andforexam#5002thepassingscoreis157.Atotalof168studentstookexam#5032andtheirscoresrangedfrom200-100withamedianscoreof185.Moststudentstakingthisexampassed.Atotalof168studentstookexam#5032andtheirscoresrangedfrom200-100withamedianscoreof185(Table7A).Moststudentstakingthisexampassed.Atotalof27studentstookexam#5002andtheirscoresrangedfrom190-157withamedianscoreof174(Table7A).Allstudentstakingthisexampassed.Forbothexams,themedianscoreishigherthanthenationalaverage.
ElementaryEducation:Reading&LanguageArts5032
9/1/2014to8/31/2015
ALL JMU
N 9,069 168
High 200 200
Low 100 100
Median 178 185
AverageRange 170-187 180-191
ElementaryEducation:Reading&LanguageArts5002
9/1/2014to8/31/2015
ALL JMU
N 8,831 27
High 200 190
Low 100 157
Median 168 174
AverageRange 159-179 166-183
19
Tables7A.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationReading&LanguageArtsTests
Tables7Bshowthequartilescoresforbothversionsofthisexam(Reading,andLanguage,Writing,andCommunicationorWriting,Speaking,andListening).Theweakestareaoftheexamistheportiondealingwithwritingforeitherversionoftheexam,with37-53%ofJMUstudentsscoringinthelowest2quartiles.
ElementaryEducation:Reading&LanguageArts5032
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd 4th(high)
Reading 11(7%) 36(21%) 56(33%) 65(39%)
Language,Writing,andCommunication 12(7%) 51(30%) 55(33%) 50(30%)
N=168
Table7B.JMUquartileresultsforElementaryEducationPraxisIIReading&LanguageArtsTest.
-ElEdMathematics
Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forexam#5033thepassingscoreis164andforexam#5003thepassingscoreis157.Atotalof168studentstookexam#5032andtheirscoresrangedfrom200-179withamedianscoreof179(Table7C).Moststudentstakingthisexampassed.Atotalof30studentstookexam#5002andtheirscoresrangedfrom200-146withamedianscoreof184(Table7A).Moststudentstakingthisexampassed.Forbothexams,themedianscoreismorethan10pointshigherthanthenationalaverage.
ElementaryEducation:Reading&LanguageArts5002
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd 4th(high)
Reading 0(0%) 7(26%) 10(37%) 10(37%)
Writing,SpeakingandListening 2(7%) 13(48%) 5(19%) 7(26%)
N=27
20
Tables7C.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationMathematicsExam.
Tables7Dshowthecontentareaquartilescoresforbothversionsofthisexam.Forallcontentareasandtestversions,morethan70%oftheJMUcohortscoredinthetop2quartilessoidentifyingtheweakestcontentareaisdifficulty.However,testversion#5003indicatesthat27%oftheJMUcohortscoredinthesecondlowestquartile.However,withonly30studentstakingthisexamitwouldbedifficulttomakeaprogrammaticchangebasedonthisresult.
ElementaryEducation:MathematicsSubtest5033
9/1/2014to8/31/2015
ALL JMU
N 11,598 168
High 200 200
Low 100 146
Median 168 179
AverageRange 153-178 171-188
ElementaryEducation:Mathematics5003
9/1/2014to8/31/2015
ALL JMU
N 9,428 30
High 200 200
Low 100 146
Median 171 184
AverageRange 157-186 171-195
21
Tables7D.JMUquartileresultsforElementaryEducationPraxisIIMathematicsExam.
-ElEdSocialStudies
Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forbothexams#5034and#5004thepassingscoreis155.Atotalof168studentstookexam#5034andtheirscoresrangedfrom200-144withamedianscoreof179(Tables7E).Moststudentstakingthisexampassed.Atotalof29studentstookexam#5004andtheirscoresrangedfrom200-150withamedianscoreof179(Tables7E).Moststudentstakingthisexampassed.Forbothversionsofthisexamthemedianscoreismorethan10pointshigherthanthenationalaverage.
ElementaryEducation:Mathematics5003
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd4th
(high)
NumbersandOperations 1(3%)
7(23%)
12(40%)
10(33%)
AlgebraicThinking 0(0%)
8(27%) 5(17%)
17(57%)
GeometryandMeasurement,Data,Statistics,andProbability 4(13%)
4(13%)
16(53%) 6(20%)
N=30
ElementaryEducation:MathematicsSubtest5033
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd 4th(high)
Number,Operations,andAlgebraicThinking 1(1%)
25(15%)
50(30%) 92(55%)
Geometry,Measurement,DataandInterpretation 5(3%)
26(15%)
81(48%) 56(33%)
N=168
22
Tables7E.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationSocialStudiesExam.
Tables7Fshowthecontentareaquartilescoresforbothversionsofthisexam.BothtestversionsindicatethatJMUstudentsstrugglewiththeareas“Geography,Anthropology,andSociology”and“WorldHistoryandEconomics”withapproximately35%and40%ofscoresinthebottomtwoquartilesrespectively.
ElementaryEducation:SocialStudiesSubtest5034
9/1/2014to8/31/2015
ALL JMU
N 10,071 168
High 200 200
Low 100 144
Median 163 176
AverageRange 155-175 166-185
ElementaryEducation:SocialStudies5004
9/1/2014to8/31/2015
ALL JMU
N 8,927 29
High 200 200
Low 100 150
Median 163 179
AverageRange 153-176 162-185
23
Tables7F.JMUquartileresultsforElementaryEducationPraxisIISocialStudiesExam.
-ElEdScience
Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.TableSet4Aofthisreportdetailsthecontentareascoveredforbothperiodsexams.Forbothexams#5035and#5005thepassingscoreis159.Atotalof166studentstookexam#5035andtheirscoresrangedfrom200-121withamedianscoreof173(Tables7G).Moststudentstakingthisexampassed.Atotalof30studentstookexam#5005andtheirscoresrangedfrom191-150withamedianscoreof170(Tables7E).Moststudentstakingthisexampassed.Forbothversionsofthisexamthemedianscorehigherthanthenationalaverage.
ElementaryEducation:SocialStudies5004
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd 4th(high)
UnitedStatesHistory,Government,andCitizenship 2(7%) 5(17%)
12(41%) 10(34%)
Geography,Anthropology,andSociology 2(7%) 9(31%)
10(34%) 8(28%)
WorldHistoryandEconomics
3(10%) 7(24%) 6(21%) 13(45%)
N=29
ElementaryEducation:SocialStudiesSubtest5034
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd
4th(high)
UnitedStatesHistory,Government,andCitizenship 6(4%)
18(11%)
66(39%)
78(46%)
Geography,Anthropology,andSociology 6(4%)
51(30%)
76(45%)
35(21%)
WorldHistoryandEconomics
14(8%)
55(33%)
55(33%)
44(26%)
N=168
24
Tables7G.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducationSocialStudiesExam.
Tables7Hshowthecontentareaquartilescoresforbothversionsofthisexam.BothtestversionsindicatethatJMUstudentsstrugglewiththeEarthScienceareawithapproximately45%ofscoresinthebottomtwoquartiles.
ElementaryEducation:ScienceSubtest5035
9/1/2014to8/31/2015
ALL JMU
N 10,036 166
High 200 200
Low 100 121
Median 167 173
AverageRange 159-176 167-182
ElementaryEducation:Science5005
9/1/2014to8/31/2015
ALL JMU
N 8,905 30
High 200 191
Low 100 150
Median 167 170
AverageRange 156-177 164-181
25
ElementaryEducation:ScienceSubtest5035
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd 4th(high)
EarthScience
16(10%) 58(35%)
38(23%) 54(33%)
LifeScience 15(9%) 37(22%)60
(36%) 54(33%)
PhysicalScience
17(10%) 34(20%)
61(37%) 54(33%)
N=166
Tables7H.JMUquartileresultsforElementaryEducationPraxisIIScienceExam.
-PreviousPRAXISIITestResultsforElementaryEducation(ElEd)ContentKnowledgeThepreviousversionsoftheElementaryContentKnowledgeexamcoversbasiccontentknowledgeacrossall4subjectareasinIdLS.Inthe2013-2014report,ETSprovideddataforboththepaperexamversionofthistestaswellastheComputerBasedTestsorElectronicversionoftheexam.Inthefollowingtext,scoresthatappearinblackrepresentthepaperexamscoresandthosethatappearasdarkorangerepresenttheElectronicexamscores.JMUstudentscontinuetodoextremelywellontheelementaryeducationcontentknowledgePraxisIItest.Themedianscorefortheprevioustestperiod(9/1/2013to8/31/2014)is177(178),whichis17(18)pointshigherthanthenationalaverage(Table7A).ThisscoreisalsohigherthanthepassscoreforVAlicensurewhichis143(143).ThelowestscoreamongallJMUstudentswhotookthetestduringthisyearwas148(147),indicatingthatallscoresforthePRAXISIItestarepassingscoresforthepreviousreportingperiod.
ElementaryEducation:Science5005
9/1/2014to8/31/2015
Subscale
Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd 4th(high)
EarthScience 2(7%)12
(40%) 8(27%) 8(27%)
LifeScience 1(3%) 9(30%) 13(43%) 7(23%)
PhysicalScience 1(3%)
11(37%) 8(27%) 10(33%)
N=30
26
ElementaryEducationPraxis2results
9/1/10to8/31/11 9/1/11to8/31/12 9/1/12to8/31/13 9/1/2013to8/31/2014
ALL JMU ALL JMU ALL JMU ALLJMU
(Electronic)
N 22,833 164 14,589 163 5,477 124 2,199 79(115)
High 200 198 200 199 200 200 200 195(198)
Low 100 143 100 134 100 143 100 148(147)
Median 162 177 163 174 163 177.5 160 177(178)
AverageRange
149-175 168-185 150-176 168-182 149-176 168-184 146-173170-181(169-183)
Table7I.PRAXISIIscoresforalltesttakersandJMUcohortforElementaryEducation
ETSreportsthedistributionofscoresforeachinstitutionrelativetothenationalquartiles.Forthepreviousreportingperiod,ScienceandMathematicshave47%(27%)and61%(43%)ofscoresinthetopquartile,respectively,whileLanguageArtsandSocialStudieshave38%(23%)and49%(28%)ofthescoresinthetopquartile,respectively(Table7B).ItisinterestingtonoteherethatthenumberofstudentsplacinginthetopquartileforallsubjectareasisSIGNFICANTLYlowerfortheComputerBasedTests.
Equallyimpressivearetheverylownumbersofstudentswhoscoredinthelowestquartiles:3%(10%)forScience,3%(4%)forMathematics,6%(2%)forSocialStudies,and5%(11%)LanguageArts(Table7B).Againitisinterestingtonotethatthenumberofstudentsscoringinthelowest(4th)quartileishigherfortheComputerBasedTestsforallareasotherthanSocialStudies.
ElementaryEducationPraxis2results
9/1/2012to8/31/2013 9/1/2013to8/31/2014
Subscale
Number(Percent)ofScoresineachquartile Number(Percent)ofScoresineachquartile
1st(low) 2nd 3rd 4th(high) 1st(low) 2nd 3rd
4th(high)
LanguageArts 8(6%) 39(31%) 39(31%) 38(31%)
4(5%)13(11%)
16(20%)18(16%)
29(37%)57(50%)
30(38%)27(23%)
27
Mathematics 4(3%) 9(7%) 39(31%) 72(58%)
2(3%)5(4%)
7(9%)21(18%)
22(28%)39(34%)
48(61%)50(43%)
SocialStudies 7(6%) 25(20%) 45(36%) 47(38%)
5(6%)2(2%)
10(13%)18(16%)
25(32%)63(55%)
39(49%)32(28%)
Science15
(12%) 24(19%) 34(27%) 51(41%)2(3%)
11(10%)15(19%)29(25%)
25(32%)44(38%)
37(47%)31(27%)
N=124 N=79N=115
Table7J.JMUquartileresultsforElementaryEducationPraxisII.
ComparingthesumofthetoptwoquartilesforeachSubscaleforthe2012-2013dataandthe2013-2014data,weseethatstudentscoresstayedtheapproximatelythesamefortheareasofLanguageArts,andSocialStudies.Therewasanimprovementof7%forMathematics,andasharpdeclinein(10%)fortheSciencearea.
28
MiddleSchoolContentAreas
TheMiddleSchoolContentAreatestsareahigh-stakesassessmentoftheconcentrationcurriculum.Studentsmustpasstwoofthesetests,usuallychosentomatchtheirtwoareasofconcentration.
MiddleSchoolLanguageArts
Therearetwoversionsofthisexamthatstudentsmayhavetakenduringthisreportingperiod.Table4BofthisreportdetailsthecontentareascoveredonthisexamforbothperiodsuptoDec.31,2013andafterJan1,2014.Forbothexams,thepassingscoreis164.Twelve(12)JMUstudentstooktheMiddleEdLanguageArtsPRAXIS2examthisyearandtheirscoresrangedfrom159to184(Table7K)andamedianscoreof169.Virginia’spassscoreforthistestis164.Moststudentstakingthistestpassed.
MiddleEdLanguageArtsPraxis2results
9/1/2012to8/31/2013 9/1/2013to8/31/2014 9/1/2014to8/31/2015
ALL JMU ALL JMU ALL JMU
N 3,177 8 725 12 3,028 12
High 200 200 200 196 195 184
Low 100 169 120 156 114 159
Median 173 179.5 171 183 164 169
AverageRange 160-184 172-188 158-184 166-190 153-171 163-178
Table7K.PRAXISIIscoresforalltesttakersandJMUcohortforMiddleEdLanguageArts.
Tables7Lshowsthequartilescoresforthesubgroupsofthisexam.Thesubscorescategorieschangedinthisyears’ETSreport,sotwotablesareincluded,onetableforthisyearandaseparatetablefortheprevioustwoyearsdata.
Table7Lshowsthat83%ofstudentsscoredinthetoptwoquartilesinLanguageUseandVocabulary.TheweakestareaisWriting,Speaking,andListeningwith50%ofthestudentsscoringinthelowest2quartilesrespectively.However,withonly12studentstakingthistest,thenumberofstudentsistoosmalltodrawreliableprogrammaticconclusions.
29
MiddleSchoolEnglishLanguageArts5047
9/1/2014to8/31/2015
Number(Percent)ofscoresineachquartile
1st(low) 2nd 3rd 4th(high)
Reading 1(8%) 4(33%) 3(25%) 4(33%)
LanguageuseandVocabulary 0(0%) 2(17%) 3(25%) 7(58%)
Writing,Speaking,andListening
1(8%) 5(42%) 1(8%) 5(42%)
EnglishLanguageArtsandInstruction
2(17%) 2(17%) 7(58%) 1(8%)
ConstructedResponse 0(0%) 5(42%) 5(42%) 2(17%)
N=12
MiddleEdLanguageArtsPraxis2Results
9/1/2012to8/31/2013 9/1/2013to8/31/2014
Number(Percent)ofscoresineachquartile
Number(Percent)ofscoresineachquartile
1st(low)
2nd 3rd4th
(high)1st(low) 2nd 3rd
4th(high)
ReadingandLiteratureStudy
0(%) 4(50%) 1(13%)3
(38%)NR NR NR NR
LanguageStudy1
(13%)2
(25%)3(38%) 2(25%) 0(0%)
5(42%)
4(33%) 3(25%)
CompositionandRhetoric
0(%) 2(25%) 2(25%) 4(50%) 1(8%) 3(25%) 3(25%) 5(42%)
ShortEssays 0(%) 3(38%) 2(25%) 3(38%) NR NR NR NR
30
LiteratureandUnderstandingText
NR NR NR NR 1(8%) 4(33%) 3(25%) 4(33%)
N=8 N=12
Tables7L.TwotablesshowingthreeyearsofJMUquartileresultsfortheMiddleSchoolLanguageArtsPraxisIIexam.Notethatsubscorereportingchangedthisyear.NRdenotesthatETSdidNotReportthesescorestoJMU.
MiddleSchoolSocialStudies
ThisexamcoverscontentinUSHistory,WorldHistory,GovernmentandCivics,Geography,Economics,andSociology/Anthropology.PassingscoreinVirginiais160.Sixstudentstookthetestin2014-15,withscoresrangingfrom155to184(Table7M).Moststudentstakingthistestpassedontheirfirstattempt.
ItisunclearwhattheBehavioralSciencescategoryrepresentedinlastyears’reportthatwasgeneratedbyETSsinceitisnotacontentcategorythatislistedforthisexam.Likewise,itisunclearwhynoShortAnswerresultsweregeneratedinthereportprovidedtoJMUasthiscontentareacountsfor25%ofthetotalscorestudentsreceiveforthisexam.InallsubscalesexceptforGovernment/Civics,atleast50%ofscoreswereinthehighesttwoquartilescomparedtothenationalaverage(Table7N).USHistoryandGovernment/CivicsaretheweakestareasforIdLSstudentswith50%ormoreplacinginthebottomtwoquartilescomparetothenationalaverage.However,withonly6studentstakingthistest,thenumberofstudentsistoosmalltodrawreliableprogrammaticconclusions.
Table7M.JMUversusUSresultsforMiddleEdSocialStudiesPraxisII.NRdenotesthatNoReportwasgeneratedforthisreportingperiod.
MiddleEdSocialStudiesPraxis2results
9/1/2012to8/31/2013 9/1/2013to8/31/2014
9/1/2014to8/31/2015
ALL JMU ALL JMU ALL JMU
N NR <5 1,615 12 2,325 6
High NR NR 200 194 200 184
Low NR NR 122 146 122 155
Median NR NR 161 168 164 166
AverageRange NR NR 152-172 158-177 152-177 162-181
31
MiddleEdSocialStudiesPraxis2Results
9/1/2014to8/31/2015 9/1/2014to8/31/2015
Number(Percent)ofscoresineachquartile Number(Percent)ofscoresineachquartile
1st(low) 2nd 3rd 4th(high)1st
(low)2nd 3rd 4th(high)
USHistory 0(0%) 7(58%) 3(25%) 2(17%) 1(17%) 2(33%) 1(17%) 2(33%)
WorldHistory 2(17%) 3(25%) 4(33%) 3(25%) 0(0%) 2(33%) 2(33%) 2(33%)
Government/Civics 0(0%) 6(50%) 4(33%) 2(17%) 2(33%) 2(33%) 2(33%) 0(0%)
Geography 3(25%) 2(17%) 6(50%) 1(8%) 2(33%) 1(17%) 2(33%) 1(17%)
Economics 2(17%) 3(25%) 4(33%) 3(25%) 0(0%) 2(33%) 3(50%) 1(17%)
ShortEssays NR NR NR NR 1(17%) 0(0%) 3(50%) 2(33%)
BehavioralSciences 0(0%) 2(17%) 5(42%) 5(42%) NA NA NA NA
N=12 N=6
Table7N.JMUquartileresultsforMiddleSchoolSocialStudiesPraxisII.NRdenotesthatNoReportwasgeneratedforthisreportingperiodorContentArea.NAdenotesthatNoAnalysiswasperformedonthiscontentareabecauseitdidnotexistduringthereportingperiod.
MiddleSchoolMathematics
AsshowninTable4Bthemathematicsexamchangedthisyear.Thirty-twostudentstookthemiddleschoolmathematicsexamduringthemostrecentreportingperiod.Theirscoresrangedfrom119to20.Themedianscorewas178,whichis9pointshigherthanthenationalaverage(Table7O).ThepassingscoreforthisexaminVirginiais165.Withtheaveragerangebeing169-187forthisyears’datamoststudentspassedthisexam.
Tables7Pshowsthequartilescoresforthesubgroupsofthisexam.Thesubscorescategorieschangedinthisyears’ETSreport,sotwotablesareincluded,onetableforthisyearandaseparatetablefortheprevioustwoyearsdata.
32
Forthecurrentreportingperiod,approximately65%ofscoreswereinthehighesttwoquartiles.(Table7H).
MiddleEdMathematicsPraxis2Results
9/1/2012to8/31/2013 9/1/2013to8/31/2014 9/1/2014to8/31/2015
ALL JMU ALL JMU ALL JMU
N 9,294 20 1,768 6 6,151 32
High 200 199 200 191 200 200
Low 100 152 118 164 118 119
Median 164 178.5 166 185 169 178
AverageRange 151-178 165-184 153-180 170-189 154-179 169-187
Table7O.PRAXISIIscoresforalltesttakersandJMUcohortforMiddleEdMathematics.
MiddleSchoolMathematics5169
9/1/2014to8/31/2015
Number(Percent)ofscoresineachquartile
1st(low) 2nd 3rd 4th(high)
ArithmeticandAlgebra 5(16%) 6(19%) 9(28%) 12(38%)
GeometryandData 2(6%) 38(25%) 11(34%) 11(34%)
N=32
MiddleEdMathematicsPraxis2Results
33
9/1/2012to8/31/2013 9/1/2013to8/31/2014
Number(Percent)ofscoresineachquartile
Number(Percent)ofscoresineachquartile
1st(low)
2nd 3rd4th
(high)1st
(low)2nd 3rd
4th(high)
ArithmeticandBasicAlgebra 3(15%) 3(15%) 6(30%)
8(40%) 0(0%) 1(17%) 2(33%) 3(50%)
GeometryandMeasurement 1(5%) 4(20%) 9(45%)
6(30%) 0(0%) 3(50%) 3(50%) 0(0%)
Functionsandtheirgraphs 1(5%) 2(10%) 9(45%)
8(40%)
1(17%) 0(0%) 3(50%) 2(33%)
Data,probability,statisticalconcepts,discretemath 2(10%) 5(25%) 6(30%)
7(35%)
1(17%) 1(17%) 3(50%) 1(17%)
Problemsolvingexercises 1(5%) 6(30%) 5(25%)
8(40%) 0(0%) 1(17%) 1(17%) 4(67%)
N=20 N=6
Table7P.TwotablesshowingthreeyearsofJMUquartileresultsfortheMiddleSchoolMathematicsPraxisIIexam.Notethatsubscorereportingchangedthisyear.
Tables7O-PcontinuetoconfirmthatstudentscontinuetobeverywellpreparedtotaketheMiddleSchoolMathematicsPRAXIS2exam.
MiddleSchoolScience
Seventeenstudentstookthistestduringtheyear.Thescoresrangedfrom141to184.ThemedianscoreforJMUstudentstakingthetestwas158comparedtothenationalaverageof156(Table7Q).ThepassingscoreforthistestinVirginiais150,whichisadropfromlastyearwherethepassingscorewas163.Moststudentswhotookthisexampassed.
In6ofthe7subscales,themajorityofscoreswereinthehighesttwoquartilescomparedtothenationalaverage(Table7J).“PhysicalSciences”and“BasicPrinciples”arethebestperformingsubscalesintheSciences.“Science,technology,society”istheweakestperformingsubscaleswith64%ofstudentsscoringinthelowest2quartiles.Ofparticularconcernisthatstudentshaveperformedpoorlyonthe“Science,technology,society”areafor7consecutiveyearsanditisbelievedthatthisismeaningful
34
resultfortheprogram.“Earth/SpaceSciences”performedpoorly3outofthelast5years,butfortheprevioustworeportingperiodsJMUstudentsscoredbetterthanthenationalaverage.
MiddleEdSciencePraxis2Results
9/1/2012to8/31/2013 9/1/2013to8/31/2014 9/1/2014to8/31/2015
ALL JMU All JMU All JMU
N 4,171 14 2,551 24 3,092 17
High 200 178 200 189 200 184
Low 100 140 109 133 100 141
Median 156 163 157 169 156 158
AverageRange 146-169 155-173 146-170 163-174 141-170 154-169
Table7Q.PRAXISIIscoresforalltesttakersandJMUcohortforMiddleSchoolSciencePraxisII.
MiddleEdSciencePraxis2Results
9/1/2013to8/31/2014 9/1/2014to8/31/2015
Number(Percent)ofscoresineachquartile
Number(Percent)ofscoresineachquartile
1st(low) 2nd 3rd4th
(high)1st
(low)2nd 3rd
4th(high)
Scientificmethodology,techniques,history 2(8%) 7(29%)
11(46%)
4(17%) 1(6%) 6(35%)
5(29%)
5(29%)
Basicprinciples 0(0%) 8(33%)9
(38%)7
(29%) 1(6%) 4(24%)6
(35%)6
(35%)
Physicalsciences 1(4%) 6(25%) 2(8%)15
(63%) 1(6%) 4(24%)7
(41%)5
(29%)
Lifesciences 1(4%) 5(21%)9 9
1(6%) 8(47%)5 3
35
(38%) (38%) (29%) (18%)
Earth/spacesciences 3(13%) 8(33%)9
(38%)4
(17%)2
(12%) 7(41%)4
(24%)4
(24%)
Science,technology,society 4(17%) 9(38%)6
(25%)5
(21%)6
(35%) 5(29%)5
(29%)1
(6%)
Shortessays 1(4%) 7(29%) 9(38)7
(29%) NA NA NA NA
N=24 N=17
Table7J.JMUquartileresultsforMiddleSchoolSciencePraxisII.
UsingDatafromTable7I,thereisperhapslittledifferencebetweentheperformanceofthe2011-2014groupsofstudentsasMedian,Low,andHighscoresareallapproximatelythesame(althoughthereisperhapsasignificantdifferenceintheLowscores).
AnalysisofPRAXISIIData
-ElementaryEducation
Inpreviousyears’reportsIdLSboasteda100%passrateforallElEdIdLSmajorsforthePRAXISIItest.ThiswasanartifactofthefactthatpreviouslytheoverallPRAXISIIscorewasanaggregationofscoresforallfoursubjectareas(Science,Math,LanguageArts,andSocialStudies)whereeachareacontributedequally(25%)tothetotalscore.Thisallowedstudentstoperformpoorlyonasingleportionoftheexamandstillpass.
Forthecurrentyears’reporttheElEdContentKnowledgetestsarehigh-stakesassessmentsinwhichstudentsmustpassallfoursubjectareaexams(Reading&LanguageArts,Mathematics,SocialStudies,andScience)inordertoreceivelicensuretoteachinVirginia.Thefollowingtable(Table7K)showspassratesofIdLSstudentsforeachofsubjectareasaftertheirfirsttestattemptandtheirfinalattempt.Additionallyfinalnotesaremadeastohowmanytimesstudentshavetakenthetest.
Section Passrateafter1sttry
FinalrateasofJune302015
Notes
LanguageArts 99%
99%
1studentfailedonce
1studentfailedtwice
36
Mathematics 93% 97% 6studentspassedonsecondtry
3studentsfailedonce
2studentsfailedtwice
SocialStudies 97%
99% 3studentspassedonsecondtry
2studentsfailedonce
Science
95%
98% 5studentspassedonsecondtry
2studentsfailedonce
1studentfailedtwice
Table7K.PassinformationfortheELEdPRAXISII.
ItisimportanttonotethatwhileeachoftheElEdpreparationareas(Science,Math,LanguageArts,andSocialStudies)mayhavesub-areaweaknesses,thefactremainsthatallareahaveatleasta97%passrate.ThereforeourIdLSstudentsareverywellpreparedtotakeeachsubjectareaofthishighstakesexamandqualifytobecomefutureteachers.
-MiddleEducation
InpreviousyearsETSsupplieddatasummarizingthenumberofattemptsattakingthePRAXISIIteststudentsneededinordertopass,howeverthisdatawasnotprovidedinthisyears’reporttous.ThefollowingtablesummarizesthenumberofattemptsthatstudentsneededtotakeindividualPRAXISIItestsinordertopassfrompreviousyear’sdata.Whilethisdataisforthevalidforthecurrentyear,wefeelitisstillaccurateinthetrendsthatitshows.
2012-2013Data 9/1/2013-8/31/2014
ContentArea Passedon1st
Attempt
Passedon2nd
Attempt
Passedon3rd(or
more)attempt
NotPassed<number
ofattempts>
Passedon1st
Attempt
Passedon2nd
Attempt
Passedon3rd(or
more)attempt
NotPassed<number
ofattempts>
Mathematics 12 -- -- -- 6 -- -- --
37
Science 6 1 2 1<2>
2<3>
14 3 3 2<1>
1<2>
1<3>
English/LanguageArts
5 -- -- -- 11(?) ?-- -?- 1<1>
SocialStudies 2 1 -- -- 5 -- -- --
Table7K.PassinformationfortheMiddleEdPRAXISII.ThereissomeuncertaintywiththisyearsreportingoftheLanguageArtscontentexam.Allthatisknownfromthedatathatwasreceivedisthatatotalof12studentstookthisexamand1studenthasyettopassthisexam.
Thisdatashowsthatinthe2012-2013cohort,91%ofthestudentswhohadtakenthePRAXISIItestshadeventuallypassedtheexams.Inthecurrentreportingperiod,89%ofmiddleeducationstudentseventuallypassedtheirPRAXISIIexams.DatasuggestthatSciencecontinuestobetheareathatstudentshavethemostdifficultypassing.However,alargemajorityofIdLSstudentstakingPRAXISIIexamspassontheirfirstattempt(74%in2008-2009,89%in2009-2010,83%in2010-2011,82%in2011-2012,and78%in2012-2013).Thereissomeuncertaintywiththisyear’sreportingoftheLanguageArtscontentexam.Allthatisknownfromthedatathatwasreceivedisthatatotalof12studentstookthisexamand1studenthasyettopassthisexam.ThereforenocalculationcanbemadeonhowmanyIdLSstudentspassedthePRAXISIIexamonfirstattempt(excludinglanguageartsfromthiscalculationwecancalculatethat72%ofstudentsfromallotherareaspassedthisexamonfirstattempt).
ST-9DATA(ItemA2,“Identifieskeyprinciplesandconceptsofsubjectmatter”)ST-9ispartofthe“AssessmentofStudentTeaching”conductedbytheCOEatJMU.Thisform(seeAppendix1),titled“PROFILEOFSTUDENTTEACHINGPERFORMANCE”isfilledoutbythecooperatingteacheranduniversitysupervisorwhiletheIdLSstudentisStudentTeaching.BoxA2ofthisform,pertainstotheabilityoftheSTUDENTTEACHERtoIDENTIFYKEYPRINCIPLESANDCONCEPTSOFSUBJECTMATTER.Ascoreof:
• 3.0meansthatthestudentteacherexplicitlyreferencesANDclearlyalignsappropriatecontentstandardswithplannedactivitiesandassessments,
• 2.0meansthatthestudentteacherexplicitlyreferencesappropriatecontentstandardsindailyplans.
• 1.0meansthatthestudentteacherinaccuratelyandvaguelyreferencesORdoesnotreferenceappropriatecontentstandards.
38
ForSpring2015,atotalof294evaluationsweremadeusingtheST-9instrumentwiththefollowingstatisticalresults(unfortunatelydataforFall2014wherenotsenttoustoincludeinthisanalysis).Duringthisreportingperiod,candidateswereratedbyasupervisorandaclassroomteacher,andtypicallydotwoplacementsperyear,thereforetheremaybeuptofourratingsforanyonecandidate.
ST-9Analysis 2009-10 2010-11 2011-12 2012-13 Fall2013 Spring2014 Spring
2015AverageScore
2.9 2.94 2.93 2.95 2.91 2.88 2.95
High 3.0 3 3 3 3 3 3
Low 1.5 1.5 1 1 2 1.5 1.5
StandardDeviation
0.26 0.22 0.22 0.17 .13 .18 .18
N 345 208 190 751 189 258 294
Table8A.ST-9scoresfor2008-2009to2013-2014.
In2008-200984%ofstudents(ElEDandMiddleSchool)scoreda3.0onitemA-2oftheST-9,meaningthat84%ofthestudentsdemonstratedthehighestlevelofmasteryofcontentknowledgeintheirclassrooms.In2009-2010,86%ofElEDandMSEDstudentsachievedthishighestlevelofmastery.FortheSpringandFallsemestersof2011,dataisavailablethatsplitstheElEDandMSEDstudentsintoseparategroups.Forthetimeperiodof2011to2012,95%ElEDmethighestlevelofmastery,while81%ofMSEDalsometthishighestlevelofmastery.Forthereportingperiod(2012-13)93%ELEDmetthehighestlevelofmastery,while78%ofMSEDalsometthishighestlevelofmasteryasdemonstratedintheclassroom.Forthisreportingperiod(Spring2015)93%ELEDmetthehighestlevelofmastery,and100%ofMSEDalsometthishighestlevelofmasteryasdemonstratedintheclassroom.
39
RESULTSFromthedatapresentedhereforthe2015-2016reportingperiod,itappearstheIdLShasmeteachofitsprogramgoals.
• FromtheCluster1,3and4dataanalysisitappearsthatthereisnosignificantdifferencebetweenIdLSandnon-IdLSstudents.Asaresult,IdLSstudentshave,asagroup,matchedotherJMUstudentsonGeneralEducationlearningoutcomes(specificallyintechnology,informationliteracy,andtheglobalandAmericanexperiences).
• FromPRAXISIIdata,itappearsthateacharea(ELED,MiddleSchoolMath,MiddleSchoolScience,MiddleSchoolEnglish,andMiddleSchoolSocialStudies)isperformingbetterthanthenationalaverages.Forthisyears’cohort,IdLSachievedapassrateof97%orbetterforElEdinall4subjectareas(Science,Math,LanguageArts,andSocialStudies).Whiledatawasnotavailabletoassessthepassrateforthisyears’cohortfortheMiddleSchoolareas,weareconfidentthatitisnotsignificantlydifferentfromthepreviousyearsforwhichdataisavailable.Asaresult,webelievethatwehavepassrateof~90%orbetterforallMiddleSchoolareas.Bothscoresarebetterthantheprogramtargetof80%.
• FromST-9data,100%ofstudentsachievedanadequatelevelofcontentproficiencyasdemonstratedintheclassroom.Thisisagainbetterthanthetargetof80%.
Whilemeetingtheseassessmentgoalsismeaningful,thereareotherrecommendationsthatwecanmakebasedonthecombinedresultsoftheseassessments.ThefollowingisalistofrecommendationstobedisseminatedtothevariousconstituenciesinIdLS.
1. AllPRAXISIIdataforthisreportingperiodneedstobecautiouslyexaminedandresultsneedtobecarefullyinterpreted.Thereare3differentPRAXISIIexamsusedforElEdassessmentinthepast2years,withthecurrenttestonlybeingusedsince7/1/15.Theexamisstillbeingnormedmeaningthatspecificresultsforthisyears’testmaynotgiveanaccurateindicationofabsolutestrengthsandweaknessesoftheprogram.
2. Sciences:ElementaryEducationPRAXISIItestresultsindicatethatstudentsarebeingadequatelypreparedtopassthisexam(Tables7Gand7H).Infactforthecurrentreportingperiod98%ofstudentspassedthisexam(Table7K)andmorethan60%ofJMUstudentswhotookthisexamplacedinthehighest2quartilesinforLifeandPhysicalSciencescomparisontothenationalaverage(Table7H).However,barelyover50%scoredinthehighest2quartilesforEarthScience.ForMiddleEducation,thecontentareaofScience,Technology,andSociety(STS)hascontinuallyshowntobeanareaofpoorperformance(Table7J).ThisistheseventhyearinarowthatSTShasbeenalowperformeronPRAXISII.Itwashopedthatanewclass(ISAT495)thatwasdevelopedsixyearsagowasgoingtohelpimprovethisarea,butwearestillseeinglowperformancenumbers(51%ofJMUstudentswhotookthistestplaceinthelowest2quartiles).InFall2013,theIdLSSteeringCommitteeconsideredthisissueintheFallof2013andbegandiscussionswithfacultyandadministratorswhodevelopedthisclassinhopesoffindingwaystoimprovestudentperformanceinthisPRAXISIIarea.Thesediscussionscontinuedintosummerof2014withtheDeanandDepartmentHeadofCISEandISAT.AsaresultitwasdeterminedthattheScience,Technology,andSocietycontentareawouldbeacoursewhosecontentwouldbedistributedacrosstheIdLSCore.
40
StudentperformanceinEarthandSpaceSciencehasvariedconsiderablyoverthelastfewyears.Thisyearstudentperformanceinthisarealookstobeweakerthanyearspast.WebelievethatthecontinueddevelopmentofanewastronomycoursestartedinAY2012-2013byDr.GearyAlbright(whichtooktheplaceofASTR301(SearchingforLifeintheUniverse))willcontinuetoimprovethecurriculumfortheMiddleEdSpaceandPlanetarySciencerequirements.Also,Dr.JenniferMangancontinuestodevelopanewWeatherandClimatecourseforIdLScoursethatwilltaketheplaceoftheexistingmeteorologycourse,againaddresstheMiddleEdlearningobjectivesforthisarea.Additionally,duringAY2013-2014,discussionswerestartedwithfacultyteachingtheEarthScienceforTeachersclasstore-examinethecontentofthiscourseanditsalignmentwithPRAXISIIlearningobjectives.WearehopefulthatthesechangeswillconcludeinAY2015-16andwillultimatelyimprovethisarea.AcrossallscienceareastheIdLSSteeringCommitteehasactivelymettodiscusseffortstore-developtheIdLScorescienceclasses(ISCI171-173).AquestionnairewascreatedanddeployedinSpring2016andiscurrentlybeinganalyzed.ResultswillbeusedinAY2015-2016torealigntheCoreSciencecurriculumwithnewNextGenerationScienceStandards.
3. SocialStudies:ItappearsstudentsarebeingwellpreparedforElementaryEducationinSocialStudieswith99%ofstudentspassingthisexam(Table7K)andmorethan60%ofstudentsplacinginthetop2quartilesnationallyforallsubareasofthisexam(Table7F).Toimprovethecurriculumevenmoreinthisarea,theIdLSsteeringcommitteeisworkingwithfacultyintheSocialStudiesareatopotentiallymodifythestructureoftheconcentrationtoinclude“tracks”whichwouldallowforamorefocusedconcentrationofcurriculumforstudents.
4. LanguageArts:ItappearsstudentsarebeingwellpreparedforElementaryEducationin
LanguageArtswith99%ofstudentspassingthisexam(Table7K)andmorethan60%oftop2quartilesnationallyforallsubareas(exceptfortest#5002areaofWriting,Speaking,andListening).BaseduponPRAXISIIdatatherehaveneverbeenanymultipleyeartrendsinthedatathatwouldindicateaconsistentlyweakpartoftheMiddleEducationprogramforLanguageArts.Toimprovethecurriculumevenmoreinthisarea,theIdLSsteeringcommitteeisworkingwithfacultyintheLanguageArtsareatodevelopmorecoursesfocusedonwriting.
5. Mathematics:Accordingtoourassessmentcriteria,themathcurriculuminIdLShasbeenthe
strongestcontentareas.AllcoursesweredesignedfromtheNCTMstandards,andthestudentsalltakethesamecoreandconcentrationcourses.ItappearsstudentscontinuetobewellpreparedforElementaryEducationinMathematicswith97%ofstudentspassingthisexam(Table7K)andthatmorethan70%ofstudentswhotookthePraxisIIElementaryContenttestscoreinthetoptwoquartilesnationally(Table7D).TherearenomultipleyeartrendsinthedatathatwouldindicateaconsistentlyweakportionoftheMiddleEducationprogramforMathematics.
PartV.Dissemination
41
TheAnnualAssessmentReportisprovidedtotheprogramdirector(FletcherLinder)anddiscussedwithbothrepresentativesfromMath/Science/TechnologyandHumanities/LanguageArtswhoserveontheIdLSExecutiveCommittee.Substantialprogresshasbeenmadeoverthepastseveralyearsandthisisanticipatedtocontinueuntilamatureassessmentprogramhasbeendeveloped.TheIdLSExecutiveCommitteereceiveanddiscussthisassessmentinformationandspecificinstrumentresultsaresharedwithrelevantareacoordinatorsandfaculty.ResultsarealsosharedwiththeCoEunitassessmentcommitteeandtheCoEAssessmentDirector(AmyThelk)aswellasseveralotherjointIdLS/CoEgroups.Finally,IdLSprovidesthisreport(orpartsofthereport)toanyotherprogramorindividualwhoexpressesinterestintheseassessmentresults.ForinstanceinthepastyearwehaveprovidedthisreporttotheDean’sofficeofCSMandCISE,DepartmentHeadsofISATandGeology,andseveralfacultyteachingintheprogram.PARTVI.UsesofEvaluation/AssessmentResultsandActionsTakenSeveral specific actions have been taken as a result of assessment results. Most of these are discussed in the previoussections.Afewofthemostsignificantactionsaresummarizedhere.
1. DatafromallpreviousyearsAPT’swereextensivelyusedinpreparingtheIdLSAcademicProgramReviewmaterials.TheIdLSexternalteamvisitedthisacademicyearandwereimpressedwithourcurriculumandourassessmentprogram.
2. TheIdLSsteeringcommitteeisworkingwithfacultyintheLanguageArtsareatomodifythecurriculumtoincludemorecoursesfocusedonwriting.
3. TheIdLSsteeringcommitteeisworkingwithfacultyintheSocialStudiesareatopotentiallymodifythestructureoftheconcentrationtoinclude“tracks”whichwouldallowforamorefocusedconcentrationofcurriculumforstudents.
4. InresponsetomultipleyearsofpoorstudentperformanceonthePRAXISIIMiddleEducationScience,sub-areaScience,technology,society,TheDirectorofIdLSandtheCoordinatorofIdLSMathematics/Science/TechnologymetwiththeDeanoftheCollegeofIntegratedScienceandEngineeringandtheDepartmentHeadofISATinSummerof2014todiscussIdLSassessmentanditimplicationsforcourserigorandimprovementofcontentfocusforthecourseISAT495.
5. WithattentionpaidtoIdLSassessment,theDirectorofIdLSandtheCoordinatorofIdLSMathematics/Science/TechnologymetwiththefacultyteachingtheScienceCorecourses(ISCI171,172,173)duringthepasttwoyears.Thesemeetingswereaimedatredesigningandre-aligningthesecoursesinpreparationfornewsciencelearningobjectivesbeingdevelopedbytheVDOE.WeanticipatethatthisworkwillconcludeduringAY2015-2016.
6. TheDirectorofIdLSandtheCoordinatorofIdLSMathematics/Science/TechnologymetwiththeDeansoftheCollegeofScienceandMathematicsinFallof2013todiscussIdLSassessmentanditimplicationsforcourserigorandimprovementofcontentfocusforScienceandMathclasses.
7. MiddleGradescurriculumwas,andcontinuestobe,revised.Specifically,inresponsetolowPRAXISIIscoresforMiddleEducationScience,sub-areaEarthandSpaceScience,anewAstronomycoursewaspilotedinFall2012.AdditionallyinFall2011,anewcourseentitledOceanographyforTeacherswastaughtinplaceofanon-teachersOceanographycourse.In2013anewcourseinWeatherandClimatewasdevelopedandtaught.FacultyteachingEarthScienceforTeachersstartedevaluatingcurriculumandwillrevisecoursecontentduringthe2015-2016academicyear.
42
8. OngoingimprovementinIdLS400basedonannualfacultydiscussionofcoursedesignandimplementation.Thisisespeciallyusefultonewfacultyandguaranteesconsistencyacrosssectionsandyears.
9. Increasedtransparencyofadvisingandscheduling,andenhancedcooperationbetweenCOEandIDLStofacilitateschedulingandsequencingofconcentrationcoursesbasedonformalandinformalsurveysofstudentsandfaculty.
10. Chemistry,STSclasses,worldhistorycourses,andmiddleeducationsciencerequirementswereallchangedinresponsetoassessmentresults.
11. IdLS400pilotedasectionwhichincludesscienceandmathematicscontentin2009-10andagainin2011-12toprovideamorefullyintegrativecontentcourseforstudents.
12. Goals,ObjectivesandMeasuresweremodifiedin2009-2010basedonpreviousyearsAPTreports.
13. ImprovedcooperationbetweenCARSandIdLStoassuredataanalysisinatimelymanner.
43
APPENDIX1.ST-9