Project TestDrive: Students and Teachers “Speak Up” about NSDL
NSDL Annual Meeting 2008Washington DCOctober 1, 2008
Julie Evans and Laurie SmithProject Tomorrow
Our discussion today:
About Project TestDrive
National Project Results and Findings
Recommendations
Next Steps
Discussion – what do you think?
NSDL Annual Meeting 2008Speak Up about NSDL
About Project Tomorrow
Our VisionPreparing today’s students to be tomorrow’s
innovators, leaders and engaged citizens
Our MissionTo support the development of critical thinking,
problem solving and creativity skills in K-12 students through innovative, research-based
uses of science, math and technology resources.
NSDL Annual Meeting 2008Speak Up about NSDL
About Project Tomorrow
NSDL Annual Meeting 2008Speak Up about NSDL
Our strengths:
Provide national leadership to improve K-12 education
Incubate and replicate school based science, math & technology programs that engage students & teachers
Leverage online tools, resources and events to build local capacities for reform and innovation
Conduct national research to inform and stimulate new discussions around education topics
About Speak Up
Annual national research projectOnline surveys - Open for all K-12 schoolsSchools/districts get back their own data for planning and budgeting
Collect data ↔ Stimulate conversationsK-12 Students, Teachers, Parents and Administrators
Inform policies & programsAnalysis and reporting – national reports, state reports, district reports Services: custom reports, consulting with districts and state agenciesNCES back end database – provide statistically significant samplings from the data
5 years of empowering authentic voices – since 2003: 1.1 million K-12 students74,000 teachers34,000 parents3,200 school leaders14,000 schools – from all 50 states, DC, American military base schools, Canada, Mexico, Australia
NSDL Annual Meeting 2008Speak Up about NSDL
1.2 million respondents
What is Project TestDrive?
National research project on the classroom effectivenessof K-12 STEM resources in the National Science Digital
Library - Supplemental to Digital Libraries go to School
Goals
– Evaluate the relevance and quality of existing resources, accessibility and impact in the classroom
– Develop rubrics for evaluating current and future NSDL content
– Develop recommendations for promoting NSDL to K-12 educators
NSDL Annual Meeting 2008Speak Up about NSDL
Why Project TestDrive?
Wealth of resources have been developed for NSDL Who is using them?How are they being used?
Extends DLgtS by…
– Modifying workshop curriculum to create JIT training for
teachers on NSDL and IA
– Additional rubrics and evaluation instruments to implement
with DLgtS workshop curriculum
– Advertises DLgtS workshops with more audiences
NSDL Annual Meeting 2008Speak Up about NSDL
Project TestDrive Methodology
Promotion to participate: + 50,000 teachers
Teacher application – screened for school diversity/mix
MOU with teachers – informed consent with students
Evaluation rubrics developed for the resources
Just in time training provided
Recommended list of resources identified
Teachers select and use 7-10 resources in classroom
Teacher and student feedback via surveys and focus groups
NSDL Annual Meeting 2008Speak Up about NSDL
Link to documents on our websitehttp://www.tomorrow.org/programs/testdrive.html
Project TestDrive Participation2007/08 School Year:
78 teacher applications approved
29 teachers used 111 resources in their classroom– 86% used at least 7 different resources
20 teachers completed online surveys
2697 students completed online surveys
11 teachers participated in focus groups
NSDL Annual Meeting 2008Speak Up about NSDL
Project TestDrive Participation
2007/08 School Year:
NSDL Annual Meeting 2008Speak Up about NSDL
Who created the resource?
0% 5% 10% 15% 20% 25%
Museums
Commercial
Fed Agencies
Prof Societies
Other
Universities
Can't determine
Project TestDrive Participation
2007/08 School Year:
NSDL Annual Meeting 2008Speak Up about NSDL
Type of resource
0% 10% 20% 30% 40% 50% 60%
Software
Audio
Datasets
Other
Video
Articles
Lesson Plans
Animation
Images
Project TestDrive Key Findings
Strategy: Outreach and Awareness Building
Strategy: Developing the Value Proposition
NSDL Annual Meeting 2008Speak Up about NSDL
Outreach and Awareness Building
How did you find the resource?
What is the most effective way to use the
resource?
How would you like to learn about resources?
How to encourage more classroom usage?
NSDL Annual Meeting 2008Speak Up about NSDL
Outreach and Awareness Building
NSDL Annual Meeting 2008Speak Up about NSDL
How did you find the resource?
0% 10% 20% 30% 40% 50% 60%
Other
Colleague
NSDL promo
Project Tomorrow
Own search
NSDL listserv
Outreach and Awareness Building
NSDL Annual Meeting 2008Speak Up about NSDL
What is most effective way to use the resource?
0% 10% 20% 30% 40% 50% 60%
Other
Assessment
Homework
Free time
Part of lab rotation
Guided practice
Extra credit
Direct instruction
Cooperative groups
Students own time
Independent practice
Intro activity
Core curriculum input
Whole class activity
Outreach and Awareness Building
NSDL Annual Meeting 2008Speak Up about NSDL
Preference for finding and learning about resources
0% 5% 10% 15% 20% 25% 30% 35%
Prof Dev Classes
Other websites
Listservs and newsletters
Curriculum Specialist
Librarian or media specialist
Other teachers
Exploring on your own
Outreach and Awareness Building
Preference for finding and learning about resources – look at who and what did not make the list!
Administrators
Students
District Technology Staff
TV / Radio Ads
NSDL Annual Meeting 2008Speak Up about NSDL
Outreach and Awareness Building
Recommendations to encourage classroom usage:
“Inside and outside” recommendations
NSDL Annual Meeting 2008Speak Up about NSDL
Outreach and Awareness Building
Recommendations to encourage classroom usage:
“Inside” recommendations:
• Increase access to school technology• Increase time to learn about and use resources• Require as part of school projects – science fairs• Funding for teachers to purchase supplies and resources
NSDL Annual Meeting 2008Speak Up about NSDL
Outreach and Awareness Building
Recommendations to encourage classroom usage:
“Outside” recommendations:
• Increase awareness – advertising, listservs• Push only selected resources – targeted approach• Bundle with textbooks and other resources• Identify key people as advocates - librarians
NSDL Annual Meeting 2008Speak Up about NSDL
Project TestDrive Key Findings
Strategy: Outreach and Awareness Building
Strategy: Developing the Value Proposition
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
Resource characteristics
Effectiveness for classroom use
Professional impact of using the resources
What do you like most/least about resources?
What should be improved?
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
How would you characterize the resource?
Top responses:
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
How would you characterize the resource?
Top responses:
Age appropriate – 70%
Enhances curriculum – 68%
Easy to use – 63%
Relevant and useful – 61%
Classroom ready / supports curriculum – 58%
Easy to integrate into curriculum – 57%
Accurate / teacher friendly – 54%
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
How would you characterize the resource?
Other responses:
Student friendly – 52%
Visually appealing – 49%
Credibility – 48%
Interactive – 45%
Clear instructions – 45%
Supports multiple learning styles – 31%
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
NSDL Annual Meeting 2008Speak Up about NSDL
Key: Very Effective Effective Not Effective
Effectiveness of the resource in the classroom
0% 20% 40% 60% 80% 100%
Student engagement
Stimulate imagination
Address learning styles
Understand standards
Higher order thinking
Knowledge application
Class participation
Challenging
Independent work
Developing the Value Proposition
Value statements about NSDL resources:
Appropriate for classroom use 88%
Content is credible 87%
Enhances current curriculum 87%
Alignment with state standards 80%
Increased student engagement 78%
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
Value statements about NSDL resources:
Appropriate for classroom use 88%
Content is credible 87%
Enhances current curriculum 87%
Alignment with state standards 80%
Increased student engagement 78%
I would use the resource again 81%
I would recommend the resource to others 79%
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
Student Views
NSDL Annual Meeting 2008Speak Up about NSDL
Student Views: Benefits of using resource activities
0% 10% 20% 30% 40% 50% 60% 70%
Use of content in job
Explore own ideas
Interaction with classmates
Liked solving problems
Increased understanding
Liked using tech
Interest in learning
Liked activities
Developing the Value Proposition
Student Views
Value statements about NSDL resources:
Often + Always
Subject is more interesting 58%
Enjoyed class participation 57%
Greater understanding of content 56%
Learned more 52%
Remember more about content 49%
More interested in subject 48%
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
Student Views
Value statements about NSDL resources:
Often + Always
I would recommend resource to a friend 52%
I would like to use the resource again 59%
I would like to use NSDL again 60%
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
NSDL Annual Meeting 2008Speak Up about NSDL
Professional impact of using resources
0% 10% 20% 30% 40% 50% 60%
Learned new assessment strategies
Enhanced interactions with students
More comfortable integratingtechnology
More comfortable with subject
Learned new engagement strategies
Would incorporate more resources
Developing the Value Proposition
What did you like most about the resource?
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
What did you like most about the resource?
Site dependent but overall comments:
Interactivity – use of animations/engaging graphics
Easy to understand and use – teacher friendly
Students could direct their own learning
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
What did you like least about the resource?
Site dependent but one general overall comment:
Depth of site content – either too much or too little
NSDL Annual Meeting 2008Speak Up about NSDL
Developing the Value Proposition
Recommendations for improving the resources
Site dependent but overall comments:
Better grade level alignment with age appropriate
activities and content
More animations/visualizations
More content OR less content
NSDL Annual Meeting 2008Speak Up about NSDL
Project TestDrive Next Steps
Further analysis of the data findings
Such as: • by contributor by resource• by contributor by grade level standard• changes in student views after increased usage• effectiveness by resource, by contributor• classroom utilization by resource• selection pathways with more exposure
NSDL Annual Meeting 2008Speak Up about NSDL
Project TestDrive Next Steps
Final report – to be published in January 2009
Marketing and outreach recommendations
Rubrics for resource evaluations
Additional data findings
Also available: Resource specific data findings for the pathways and contributors
NSDL Annual Meeting 2008Speak Up about NSDL
Want more information?
Speak Up teacher data on digital resources
• what characteristics are most important to you when choosing online resources?
• how have you used online resources in the past 12 months?
• what factors influence your inclusion of 21st century tools in your science classroom?
NSDL Annual Meeting 2008Speak Up about NSDL
More Speak Up? www.tomorrow.org
Webinars – new series
Podcasts
Speak Up Blog
Sign up for news alerts
Customized reports and consulting services
Conference presentations/workshops
Speak Up 2008
NSDL Annual Meeting 2008Speak Up about NSDL
More Speak Up findings?
Visit www.tomorrow.org
Student, Teacher, Parent & Administrator Data Findings
Reports such as:
Learning in the 21st Century: A National Report of Online Learning(Oct 2007, Updated Jun 2008)
Inspiring the Next Generation of InnovatorsStudents, Teachers and Parents Speak Up about Science Education
(June 2008)
NSDL Annual Meeting 2008Speak Up about NSDL
More Speak Up?
Upcoming Presentations and Events:
Innovative Learning Conference 2008 October 14-16, 2008San JosePresentation: Speak Up data on mobile devices
NSBA T+L 2008October 28-30, 2008SeattlePresentation: Administrator data from Speak Up Release of new report: “New Visionary Administrator”
NSDL Annual Meeting 2008Speak Up about NSDL
Surveys for K-12 students, teachers, parents and administratorsSurvey for parents in English and SpanishNo fee or charge to participate
100% online – each survey only takes 20 minutes100% confidential; password protected
Questions regarding:teaching and learning with technology, 21st century skills, professional development, science instruction, STEM careers, emerging technologies and online learning
New question themesBroadband access/issues – home and at schoolDigital contentMedia and information literacy
Surveys open from Oct 20 thru Dec 19Schools/districts get data back in FebruaryNational data findings released in Washington DC in spring 2009
NSDL Annual Meeting 2008
Speak Up 2008
Thanks! If you have any questions, please contact:
Julie [email protected]
949-609-4660 x15
Laurie [email protected]
949-609-4660 x17
NSDL Annual Meeting 2008Speak Up about NSDL