observing the sun at mcdonald...

33
Evaluation Report Observing the Sun at McDonald Observatory Executive Summary Background: This project funded April 2005-March 2007 through IDEAS developed and provided programming for using equipment donated to McDonald Observatory’s Visitor Center to provide students with first-hand experiences of observing the sun, both those who physically visit the Observatory and those who make an electronic visit via an individual classroom videoconference. Strategy: The recent availability of videoconference equipment in every Texas school district together with the donation of equipment to the Observatory's Visitor Center allowed McDonald Observatory’s Visitor Center to begin an innovative program of communication with classrooms and schools using the power of technology to “to do things in education that we were previously unable to do.” Project Evaluation Methodology included 1) documentation of formative feedback about qualities of the activities and materials and 2) interviews with teachers to evaluate use and perceived qualities and effects of the videoconferences. Results and Conclusions: Members of the project team and Advisory Board provided formative feedback using the structured assessment tool to report the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. o As a set, the eight products—e.g., “Conference DVD segments on observatory tour/solar telescope operation/careers”—were evaluated to be strongly consistent with the 5E model. All of the products were indicated to be at least moderately strong across the 5Es. Strength was especially notable for Engage, Explain, and Evaluate. o Inspection of item responses for attributes directly descriptive of the project’s stated objectives for the programming showed each of the products with exceptional Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 1 of 33

Upload: others

Post on 30-Jun-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Evaluation Report

Observing the Sun at McDonald Observatory

Executive Summary

Background: This project funded April 2005-March 2007 through IDEAS developed and provided programming for using equipment donated to McDonald Observatory’s Visitor Center to provide students with first-hand experiences of observing the sun, both those who physically visit the Observatory and those who make an electronic visit via an individual classroom videoconference.

Strategy: The recent availability of videoconference equipment in every Texas school district together with the donation of equipment to the Observatory's Visitor Center allowed McDonald Observatory’s Visitor Center to begin an innovative program of communication with classrooms and schools using the power of technology to “to do things in education that we were previously unable to do.”

Project Evaluation Methodology included 1) documentation of formative feedback about qualities of the activities and materials and 2) interviews with teachers to evaluate use and perceived qualities and effects of the videoconferences.

Results and Conclusions:Members of the project team and Advisory Board provided formative feedback using the structured assessment tool to report the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning.

o As a set, the eight products—e.g., “Conference DVD segments on observatory tour/solar telescope operation/careers”—were evaluated to be strongly consistent with the 5E model. All of the products were indicated to be at least moderately strong across the 5Es. Strength was especially notable for Engage, Explain, and Evaluate.

o Inspection of item responses for attributes directly descriptive of the project’s stated objectives for the programming showed each of the products with exceptional strength relative to at least one of the stated objectives. Across the set of products, there was exceptional strength documented for each of the stated program objectives that were paraphrased on the assessment tool.

The seven teachers who participated in the interviews brought their students to the Live from McDonald Observatory Videoconference “Observing the Sun” from all levels of schools in small towns all over Texas and beyond.

o The most frequent theme in the teachers’ responses to the interview questions was that they “loved” the Observing the Sun videoconference experience and/or perceived it to be effective.

o The teachers indicated awareness of and access to the pre- and post-visit suggestions and materials, but they made relatively more use of the pre- than of the post-visit resources.

o Teachers indicated the videoconferencing technology enabled them do things in education they were previously unable to do—e.g., “I love technology and incorporating technology…It’s about as good as it gets to have an astronomer to literally teach your class one day!”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 1 of 19

Page 2: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Observing the Sun at McDonald Observatory

Goals of this project funded April 2005-March 2007 through IDEAS were to:o develop and provide effective programming on the sun for K-12 students and teachers

who visit McDonald Observatoryo develop and provide effective programming on astronomy, including solar viewing, for

students and teachers who make visits to McDonald Observatory electronically through individual classroom videoconferences

o create a team of teachers, science education staff, and scientists to develop these materials

o share both the content and the process of creating these programs with other observatories involved in similar outreach programs

Context: During 2005, several donations of equipment were made to expand and enhance the K-12 educational science program at the McDonald Observatory Visitors Center in Fort Davis, Texas. This project developed the programming for using the donated equipment to help teachers address major deficiencies that students experience with items in the Texas Assessment of Knowledge and Skills (TAKS) about characteristics of the Sun and other objects in the sky. Through this programming students are provided with first-hand experiences of observing the Sun, both those who physically visit the Observatory and those who make an electronic visit via an individual classroom videoconference.

Objectives for the programming were stated as follows:o Students discover connections between astronomy and their school’s science curriculum.o Students learn about the relationship between science and technology through contact

with instruments and their use.o Facilitator(s) engage students’ prior knowledge and understanding based on pre-visit

materials.o Facilitator acts as a co-investigator with the students and employs guided inquiry to focus

students’ attention on driving questions and concepts.o Students reflect on the experience through use of post-visit materials.

Strategy: Activities and supporting materials for student field experience were developed consistent with the 5E model for promoting student inquiry as a process for learning. The project engaged an advisory board of teachers, science education staff, and scientists and used feedback from formative evaluation to help find the right mix of video segments, interactive segments, and engaging questions that promote student response so that students who participate in Observing the Sun via video conference don’t feel they are just “watching TV.” The recent availability of videoconference equipment in every Texas school district together with the donation of equipment to the Observatory's Visitor Center allowed McDonald Observatory’s Visitor Center to begin an innovative program of communication with classrooms and schools using the power of technology to “to do things in education that we were previously unable to do.”1

Products developed included Pre-conference/Pre-visit sun assessment, Demonstrations (on DVD), Live (and pre-recorded) images of the sun and commentary, Viewing the sun with solar telescope, and Post-visit suggestions. Products specific to Student Field Experience were Use of astroscan and the Large sun and word wall. Products specific to videoconference were DVD segments on observatory, conference hand-outs for students, and conference teacher guides.

1 O’Neil, H. & Perez, R. (2003). Technology Applications in Education. NJ: Lawrence Erlbaum Associates

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 2 of 19

Page 3: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 3 of 19

Page 4: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Evaluation Methods

Two evaluation studies were conducted to: (1) help improve the project; and (2) communicate results to a variety of stakeholders. The study protocol was approved by the Institutional Review Board (IRB) of the University of Texas at Austin.

Study 1 provided documentation of formative feedback about the activities and materials developed for “Observing the Sun at McDonald Observatory.”

o Design: In spring 2006 and again in spring 2007, the project team and the Education Advisory Board used a structured assessment tool to evaluate extent to which each product (e.g., video segment) was consistent with program objectives and requirements (e.g., has features to help students discover connections between astronomy and their school’s science curriculum).

o Instruments: The assessment tool was adapted from narrative presented in the “Observing the Sun” project description and from checklists and rubrics for assessing consistency with the 5E model for inquiry-guided learning.2

o Analyses: Multiattribute evaluation (MAE) procedures were applied to aggregate the data. Separate reports were prepared for each product. A summary report then was prepared to document strengths and recommend improvements across program components. The appendix to this final evaluation report includes a copy of the assessment tool and the separate reports compiled from the data collected spring 2007.

Study 2 used qualitative methods to evaluate acceptance, use, and perceived effectiveness of the program.

o Design: Interviews were conducted with teachers who participated in the videoconference programs during the period April 2006 through January 2007. We attempted an interview with each of the 12 teachers who participated in “Live from McDonald Observatory – Observing the Sun” videoconferences with their students during the selected time period and completed interviews with 7. We did not interview teachers who participated with their students on-site at McDonald Observatory because weather and other conditions resulted in only 3 such teachers for the selected time period.

o Instruments: The interviews were conducted by telephone. Questions asked where the observatory experience fit into the class curriculum, how students prepared for their experience with the observatory, whether and how the experience helped students learn about and have contact with scientific technology and instruments, and what kinds of follow-on activities or consequences there were. The appendix to this final evaluation report includes a copy of the script for the interview.

o Analyses: With the participant’s permission, the telephone interviews were recorded, transcribed, and coded for thematic analysis.

2 The underlying rubrics are available to view on line (www.unc.edu/destiny/5Es.htm and www.thetrc.org/edu/doc/5E/5ERub0304.pdf ). The 5E Model is attributed to Bybee R (1997) Achieving Scientific Literacy: From Purposes to Practices, Heinemann: Portsmouth, NH.

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 4 of 19

Page 5: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Results of Study 1

Strengths of Products Developed for “Observing the Sun”

Members of the project team and Advisory Board provided formative feedback using the structured assessment tool to report the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. During February and March 2007, assessments were submitted by 9 individuals. Several of the respondents made narrative comments that their evaluations in March were based on the “revised” descriptions and materials.

The 5E phases are Engage, Explore, Explain, Elaborate, and Evaluate. The assessment tool is multi-attribute. The four attributes listed for the Engage phase, for example, are “creates interest,” “generates curiosity,” “asks questions,” and “elicits responses from students that uncovers what they know and think.” Attribute response options are “strongly agree,” “agree,” “disagree,” and “disagree strongly.” Because some of the phases have five attributes and some only four, MAE procedures recoded the item responses such that “strongly agree” = 100, “agree” = 67, “disagree” = 33, and “disagree strongly” = 0. Missing responses were not included in the aggregate scores calculated across attributes. Aggregate scores in the range 64 to 76 were accepted as evidence of moderate strength; 77 to 89 indicated strength; and 90 to 100 indicated exceptional strength.

As a set, the products were evaluated to be strongly consistent with the 5E model. All of the products developed in the “Observing the Sun” project were indicated to be at least moderately strong across the 5Es. “Handouts for student use during the experience,” “DVD of demonstration video clips”, and “Post-visit suggestions” were products indicated to be at least moderately strong in each of the 5E phases.

Strength was especially notable for Engage, Explain, and Evaluate across all of the products. The “DVD of demonstration video clips” and the activity and materials for “Show earth’s rotation using astroscan telescope” were indicated to have exceptional strength for Engage. Elaborate was the only phase where strength was not common across the activity descriptions and supporting materials developed for the “Observing the Sun” project.

Inspection of item responses for attributes directly descriptive of the project’s stated objectives for the programming showed each of the products with exceptional strength relative to at least one of the stated objectives. Across the set of products, there was exceptional strength documented for each of the stated program objectives that were paraphrased on the assessment tool. Additional detail can be viewed in the individual product reports (Appendix B).

Strength in addressing Objectives as stated for the programming

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 5 of 19

Page 6: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Results of Study 2

Interview respondents were generally representative of the population of teachers who participated with their students in “Observing the Sun” Videoconferences

April 2006 through January 2007Date of

ConferenceGrade Level

TaughtStudents

Participating Location of the SchoolInterview

CompletedSpring 07 Elementary 16 Central TX √Spring 07 Elementary 45 North TXSpring 07 Elementary 26 West TX √Spring 07 High School 18 Central TXSpring 07 High School 26 Central TXSpring 06 High School 30 East TX √

Fall 06 High School 16 Outside TX √Spring 07 Middle/JH 45 Central TXSpring 07 Middle/JH 28 Central TX √

Fall 06 Middle/JH 45 Central TX √Spring 07 Middle/JH 48 North TX √Spring 06 Middle/JH 30 South TX

Teachers who participated in the interviews, like the population of teachers whose students participated in Live from McDonald Observatory Videoconference “Observing the Sun” during the period April 2006 through January 2007, came from all levels of schools all over Texas and beyond. Attempts were made to reach all of 12 of the teachers whose students participated in the Videoconferences Live from McDonald Observatory. All 12 were from schools located in small or very small towns. We were unable to reach 4 of the teachers by telephone. One teacher declined the interview.

The time and thoughtful responses the teachers gave to help evaluate the “Observing the Sun” project and identify options for continuing to improve the reach and the effectiveness of McDonald Observatory’s student field experiences in science and technology is very much appreciated. We promised to keep the interviews brief. The teacher’s time on the phone ranged from 7 to 20 minutes. The briefest interview was with a language arts teacher who had taken her class to the scheduled videoconference in lieu of the science class originally scheduled that, instead, went to a science camp that day.

The most frequent theme in the teachers’ responses to the interview questions was that they “loved” the Observing the Sun videoconference experience and/or perceived it to be effective. They said: “It was excellent. I really enjoyed it and I learned from it and the kids enjoyed it…It was an awesome experience for this kids …” “The information was very beneficial.” “The kids were tickled…they loved it.” “They were really excited…you could see it on their face…maybe this is something I could be interested in…you know, for their future.” “It was exciting for the students…they learned a lot…it was very unique…it was great!” “It was good and I would definitely do it again.” “I really enjoyed it and the kids really enjoyed it…I think that field trip really helped them to understand and they’ll remember it better…If that question comes up on the TAKS, they’ll remember, ‘I had that field trip.’”

The fit with TAKS was another frequent theme. One of the teachers said, “We brought in the 4th grade so they will be ready for the 5th grade TAKS…” Another said, “There are questions about the sun on the TAKS, even if its not a lot of questions…[this experience] actually answered some questions.” Others said: “It was for my science class. We had been talking about the solar system. This was able to fit it and explain more. Also it was part of TAKS preparation;” “…the

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 6 of 19

Page 7: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

sun and how it fit into the different stars…one of our TEKS is star life cycle;” and “We had studied astronomy and as you know astronomy is part of the TEKS in 8th grade…”

o The most frequent theme in the teachers’ responses to the interview questions was that they “loved” the Observing the Sun videoconference experience and/or perceived it to be effective.

o For some, the videoconference was a planned enrichment for their advanced placement and gifted and talented students. For others, it was a culminating activity for a science unit. And for others, it was an opportunity that developed at the “last minute.”

o Irrespective of how or why they elected to participate, the teachers found many positive aspects of the experience and were especially impressed with the interactive capabilities of the video technology.

o The teachers indicated awareness of and access to the pre- and post-visit suggestions and materials, but they made relatively more use of the pre- than of the post-visit resources.

o A final theme touched on how the videoconferencing technology enabled the teachers to do things in education they were previously unable to do.

Although teachers provided these kinds of strong endorsements of the programming and of its relevance in their science instruction, another common theme in the interview responses was the sometimes opportunistic circumstances that brought their class to the “Observing the Sun” videoconference. The last minute or happenstance nature of the event may have constrained its capabilities for promoting student learning. Three of the 7 teachers made comments such as “We were not currently talking about space or solar system, but it is something we have talked about in the past. When the opportunity became open, I wanted [the students] to be able to see it…It was a time slot that another school had set up and I just took the class that I had at that time. It was extremely last minute…the students found out only about 5 minutes before we went.” Another said, “Our 5th graders were on a field trip that day, we brought in the 4th grade …and we had a 6th grade classroom in there too… We didn’t really interact or anything…We were just kind of watching…The science teacher wasn’t there for the video...I’m a language teacher, so...” And the third said, “They were not as prepared as I would have liked for them to have been because this was a last minute deal.”

For some of the teachers, the videoconference was a planned enrichment for their advanced placement and gifted and talented students (GTP/AP). One said, “I have five classes of GTP/AP students it just so happened that the time slots that were available were with classes that I thought could benefit a lot…” Another said, “I wanted my gifted students and the higher kids to do something different…We just grabbed students from all over the 4th grade that were excelling and we wanted to give them something extra.” Others took students to the videoconference because, “It was a nice end to our unit…a culminating experience.”

Irrespective of how or why they elected to participate, the teachers found many positive aspects in the experience and were especially impressed with the interactive capabilities of the video technology and the way it brings the observatory experience to students who might otherwise never visit an observatory.

o One said, “I was surprised how impressed the kids were. I think of these kids as being technology literate, and they were so surprised that [the facilitator] could see them and

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 7 of 19

Page 8: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

talk to them. …In their comments, the kids said to him things like ‘I am the one sitting in front in the red shirt.’ It was like the contact with [the facilitator] was so personal….They really felt like he was there with them.”

o Another said “We had never done a distance learning with that group of 6th graders, so just seeing they could actually talk to someone that was somewhere else and ask questions and get answers was real eye-opening for them….We live in a rural area, and so it was very good they got to have that experience…They were able to see the telescope and talk to him… everyone at some point got to ask questions and answer questions…It was very good.”

o Another said, “[The students] were able to answer questions when [the facilitator] asked about the black spots on the sun…The nice thing was he was able to expand, show them an example—His “hands-on” to show why the sun has sun spots…That made it more clear to the kids what a sunspot actually was.”

o Another said, “[The facilitator] was great in getting the kids involved…He had questions for them and they had questions for him and it was just great, interactive...Kept it going, back and forth.”

o Yet another commented, “This was the very first video teleconference our school had had…The students knew that and the district sent people out weeks ahead and we had sent it out to the news media….[The students] knew they were participating in something special, and that was a real experience with technology in itself…They were also able to see the instruments [the facilitator] showed them….They were very interested as he walked around the observatory and pointed out different parts of the telescope that we had talked about in our unit on the foundations of astronomy….He had the students engaged the entire time.”

o And another indicated that “although we had a lot of technical difficulties the day of the conference…just the exposure of seeing the observatory, seeing the various instruments, exposure to having a videoconference, and [the facilitator] doing demonstrations on the sun, showing sun spots” were examples of student learning from the experience.

o And the last said, “[The facilitator] was able to show the actual activity [and] they were able to actually see…It was like hands-on…they weren’t actually touching it, but they were watching him do the investigative part of it.”The teachers indicated awareness of and access to the pre- and post-visit suggestions and

materials, but they made relatively more use of the pre- than of the post-visit resources. One teacher said, “I didn’t have a lot of follow-up because afterwards we just went into the next project….We did watch the intro DVD as a pre-program…We had studied astronomy and we had done some of the activities to prepare for the presentation. It was more of a culminating event than it was in the curriculum because when we were done with it we immediately went on to something else…One of the things that was so positive is that the videoconferences was such a motivating statement for the activities we did with the kids [before the conference]….It motivated the kids in preparation for the conference…They studied hard for it.” Another commented, “After the experience, I was in the middle of teaching health and so, to be honest, I have not done a lot with the information….But I do have all the materials it suggested we do for the follow-up.” Another said, “We just finished our talking about the solar system a little more, and then we had an assessment, and then we had to move on to something else….But we had done the pre-activities they had on the website….They did a drawing of the sun and then they were able to see and to talk [to the facilitator] about how their drawings were similar to or different from what they actually saw and talked about.” Another commented, “The observatory had a link on their site and I could get those activities…[The students] developed their own questions before the conference…We missed the first 3 days of the following week because of an ice storm and we never went back and did any follow-up…We were meaning to do that…”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 8 of 19

Page 9: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

One of the teachers did, however, report “There were some follow-up activities the observatory had sent and we participated in…We also went to the observatory website ‘What are Astronomers Doing?’ and they clicked to look what different astronomers were there when we were doing the conference and what kind of research they were doing…They were very interested in the work about ‘The extra solar planets’…They were pretty excited about that.”

A final theme touched on how the videoconferencing technology enabled the teachers to do things in education they were previously unable to do. One teacher said, “[the facilitator] was able to explain in more depth [than I could]…he was able to answer right off the top of his head where they could understand it better…We couldn’t actually go to the observatory, but [the students] could get the information from the observatory [where] they are more expert and could answer questions that I couldn’t answer.” Another said, “From the area that we’re coming from, it is low SES kids and so a lot of them will not be able to go to an observatory or anything like that….It gave them an opportunity to go to the observatory without leaving the classroom…they were just really excited.” Another noted, “We are so far from the observatory…It is not like we’ll ever be able to do a field trip…But this way, they were able to have the experience, see the sun through the telescope.” Another said, “Just to know the observatory is there…It is within driving distance—even though they might not go there—just exposure to what an observatory does.” A nice conclusion to this report is the observation of the teacher who said, “I love technology and incorporating technology…It’s about as good as it gets to have an astronomer to literally teach your class one day!”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 9 of 19

Page 10: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Appendix A – Instruments

Please give us your opinions about characteristics of the program element named:

“ ___________________________________________ “

Stro

ngly

Ag

ree

Disa

gree

Stro

ngly

Creates interest

Generates curiosity

Asks questions

Elicits responses from students that uncovers what they know and think

Uses students’ previous experiences as a basis for explaining concepts

Provides opportunities for students to describe objects and events

Encourages students to ask questions

Presents problem(s) for students to solve

Encourages students to work together

Allows students to connect astronomy to their school’s science curriculum

Uses contact with instruments to connect science and technology

Encourages students to explain concepts in their own terms

Asks students for justification or evidence to support their explanations

Encourages students to communicate their explanations to others

Encourages students to test their explanations

Reminds students that there are alternative explanations

Encourages student to reflect on their observatory experience

Refers students to data as a resource for elaborating what they know

Builds critical and logical thinking skills

Allows the teacher to observe and assess students’ knowledge and skills

Allows students to assess their own learning

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 10 of 19

Page 11: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

Telephone script for interview with teachers

Hello. My name is xxxxx with the Observing the Sun project at McDonald Observatory. I’m calling to ask you to help us identify ways to ensure that student experiences with the observatory are as beneficial as possible. We appreciate that you have agreed to participate in this evaluation. With your permission, I am going to record this interview. I have six questions to ask and expect that the interview should not take more than about 20 minutes. Is this agreeable to you?

These questions are about the experience of the group of xxxx students in grades xxxx that (came to the observatory) (participated in the videoconference) on xxxxx.

1. Where did the observatory experience fit into your curriculum? (prompts: Did it serve to introduce a unit on the sun? OR help teach basic content? OR provide a culminating activity? OR provide extracurricular enrichment? OR serve some other purpose?)

2. How did you decide which students to bring to the observatory experience? (prompts: Was it an entire classroom? Or was it a select group of students? If select, how were they selected?)

3. How did students get prepared for their experience with the observatory? (prompts: Were there particular materials or resources that helped students get ready? If so, what were they? What are some examples of things that happened (at the observatory) (during the videoconference) that let you know the students were OR were not well prepared for the experience?)

4. In what ways did the event help your students learn about and have contact with scientific technology and instruments? (prompts: Did you observe changes in student understanding? If so, please give me an example. Did the event give your students the opportunity to explore “guided inquiry” to frame research questions? If so, please give me an example? How well did the facilitator engage students in a “co-investigator” role?)

5. Tell me a little about what happened during the weeks after the experience with McDonald Observatory. What kinds of follow-on activities or consequences were there? (prompts: Did the post-visit materials or resources help students link their observatory experience to their classroom learning and/or help them to review and reflect on the event? If so, what are some examples? What did the students talk about? If no, what would have been helpful for linking the observatory experience with classroom learning?)

6. Would you do this (conference) (observatory visit) again? Why or why not?

THANKS very much for your help. Please feel free to call Cindy Roberts-Gray at 409/762-2499 if you have other comments or questions about this evaluation of the Observing the Sun project.

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 11 of 19

Page 12: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Show the earth’s rotation using astroscan telescope to project image of the sunTotal 5E score: 79 on 100 point scale

Creates interest 100Engage Generates curiosity 100

98 Asks questions 100Elicits responses from students that uncovers what they know and think 91

Provides opportunities for students to describe objects and events 90Explore Encourages students to ask questions 95

88 Presents problem(s) for students to solve 91Encourages students to work together 67Uses contact with instruments to connect science and technology 95

Uses students’ previous experiences as a basis for explaining concepts 90Explain Allows students to connect astronomy to their school’s science

curriculum76

82 Encourages students to explain concepts in their own terms 81Asks students for justification or evidence to support their explanations 81

Encourages students to communicate their explanations to others 67Elaborate Encourages students to test their explanations 57

53 Reminds students that there are alternative explanations 43Refers students to data as a resource for elaborating what they know 43

Encourages student to reflect on their observatory experience 52Evaluate Builds critical and logical thinking skills 81

72 Allows the teacher to observe and assess students’ knowledge and skills 81Allows students to assess their own learning 72

Exceptional strength of the “Show earth’s rotation using astroscan telescope to project an image of the sun” is its capabilities for engaging the learner (score of 98 on the 100 point scale). Additional strengths are the product’s capabilities for promoting the learner’s own involvement in the Explore, Explain, and Evaluate phases of the 5E model for promoting student inquiry as a process for learning (scores of 88, 82, and 72).

Elaborate (score of 53) is a phase of the 5E model where the product could be strengthened and/or where it needs support of other components of the “Observing the Sun at McDonald Observatory.”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 12 of 19

Page 13: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Conference DVD segment on observatory tour, telescope, and careersTotal 5E score: 69 on 100 point scale

Creates interest 92Engage Generates curiosity 96

83 Asks questions 79Elicits responses from students that uncovers what they know and think 63

Provides opportunities for students to describe objects and events 63Explore Encourages students to ask questions 79

68 Presents problem(s) for students to solve 54Encourages students to work together 46Uses contact with instruments to connect science and technology 100

Uses students’ previous experiences as a basis for explaining concepts 75Explain Allows students to connect astronomy to their school’s science

curriculum83

70 Encourages students to explain concepts in their own terms 63Asks students for justification or evidence to support their explanations 58

Encourages students to communicate their explanations to others 54Elaborate Encourages students to test their explanations 50

53 Reminds students that there are alternative explanations 46Refers students to data as a resource for elaborating what they know 63

Encourages students to reflect on their observatory experience 71Evaluate Builds critical and logical thinking skills 71

71 Allows the teacher to observe and assess students’ knowledge and skills 71Allows students to assess their own learning 71

Strength of the “Conference DVD segment on observatory tour, solar telescope operation, and careers” is its capabilities for engaging the learner (score of 83 on the 100 point scale), especially its capabilities for generating curiosity (scores of 96 on the 100 point scale). The product also was evaluated to have moderate capabilities for involving the learner in the Explore, Explain, and Evaluate phases of inquiry-based learning (scores of 68, 70, and 71).

There is considerable room for improvement and/or need for support from other products or components of the “Observing the Sun at McDonald Observatory” project in Elaborate phase of the 5E model for promoting inquiry as a process for learning.

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 13 of 19

Page 14: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted in blue are paraphrase of statements of the project objectives.

Product: DVD of demonstration video clips, Teacher/Facilitator resourcesTotal 5E score: 77 on 100 point scale

Creates interest 92Engage Generates curiosity 92

94 Asks questions 96Elicits responses from students that uncovers what they know and think 96

Provides opportunities for students to describe objects and events 79Explore Encourages students to ask questions 84

73 Presents problem(s) for students to solve 79Encourages students to work together 46Uses contact with instruments to connect science and technology 75

Uses students’ previous experiences as a basis for explaining concepts 96Explain Allows students to connect astronomy to their school’s science

curriculum79

86 Encourages students to explain concepts in their own terms 88Asks students for justification or evidence to support their explanations 79

Encourages students to communicate their explanations to others 75Elaborate Encourages students to test their explanations 63

67 Reminds students that there are alternative explanations 75Refers students to data as a resource for elaborating what they know 54

Encourages student to reflect on their observatory experience 54Evaluate Builds critical and logical thinking skills 71

67 Allows the teacher to observe and assess students’ knowledge and skills 71Allows students to assess their own learning 71

Exceptional strength of the “DVD of demonstration video clips, Teacher/Facilitator resources” is its capabilities to Engage the learner (score of 94 on the 100 point scale). Another strength is the product’s capabilities for promoting the learner’s own involvement to Explain concepts (score of 86). The product also was evaluated by the project team as having moderate capabilities for involving the learner in the Explore, Elaborate, and Evaluate phases of the 5E model for promoting inquiry as a process for learning.

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 14 of 19

Page 15: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Handouts for student use during the conference or visitTotal 5E score: 76 on 100 point scale

Creates interest 82Engage Generates curiosity 85

86 Asks questions 85Elicits responses from students that uncovers what they know and think 93

Provides opportunities for students to describe objects and events 100Explore Encourages students to ask questions 82

85 Presents problem(s) for students to solve 89Encourages students to work together 78Uses contact with instruments to connect science and technology 74

Uses students’ previous experiences as a basis for explaining concepts 93Explain Allows students to connect astronomy to their school’s science

curriculum85

65 Encourages students to explain concepts in their own terms 89Asks students for justification or evidence to support their explanations 82

Encourages students to communicate their explanations to others 74Elaborate Encourages students to test their explanations 59

64 Reminds students that there are alternative explanations 52Refers students to data as a resource for elaborating what they know 70

Encourages students to reflect on their observatory experience 74Evaluate Builds critical and logical thinking skills 78

81 Allows the teacher to observe and assess students’ knowledge and skills 93Allows students to assess their own learning 78

Strengths of the “Handouts for students’ use during the conference or visit” are its capabilities to Engage the learner (score of 86 on the 100 point scale) and involving the learner in the Explore and Evaluate phases of inquiry-based learning (scores of 85 and 81). The product also was evaluated to have moderate strength in the Explain and Elaborate phases of the 5E model for promoting inquiry as a process for learning (scores of 65 and 64).

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 15 of 19

Page 16: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Large sun and word wallTotal 5E score: 75 on 100 point scale

Creates interest 81Engage Generates curiosity 86

86 Asks questions 81Elicits responses from students that uncovers what they know and think 95

Provides opportunities for students to describe objects and events 95Explore Encourages students to ask questions 86

78 Presents problem(s) for students to solve 86Encourages students to work together 81Uses contact with instruments to connect science and technology 43

Uses students’ previous experiences as a basis for explaining concepts 95Explain Allows students to connect astronomy to their school’s science

curriculum86

80 Encourages students to explain concepts in their own terms 81Asks students for justification or evidence to support their explanations 57

Encourages students to communicate their explanations to others 72Elaborate Encourages students to test their explanations 52

56 Reminds students that there are alternative explanations 43Refers students to data as a resource for elaborating what they know 57

Encourages student to reflect on their observatory experience 57Evaluate Builds critical and logical thinking skills 62

73 Allows the teacher to observe and assess students’ knowledge and skills 91Allows students to assess their own learning 81

Strengths of the “Large sun and word wall” are its capabilities to Engage the learner (score of 86 on the 100 point scale) and promote the learner’s own involvement in the Explore and Explain phases of inquiry-based learning (scores of 78 and 80). The product also was evaluated by the project team to have moderate capabilities for promoting in the Evaluate phase (score of 73).

Elaborate (score of 56) is a phase of the 5E model where the product could be strengthened and/or where it needs support of other components of the “Observing the Sun at McDonald Observatory.”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 16 of 19

Page 17: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Post-visit suggestionsTotal 5E score: 80 on 100 point scale

Creates interest 79Engage Generates curiosity 75

84 Asks questions 88Elicits responses from students that uncovers what they know and think 92

Provides opportunities for students to describe objects and events 84Explore Encourages students to ask questions 75

73 Presents problem(s) for students to solve 67Encourages students to work together 71Uses contact with instruments to connect science and technology 67

Uses students’ previous experiences as a basis for explaining concepts 92Explain Allows students to connect astronomy to their school’s science

curriculum79

81 Encourages students to explain concepts in their own terms 79Asks students for justification or evidence to support their explanations 75

Encourages students to communicate their explanations to others 88Elaborate Encourages students to test their explanations 54

72 Reminds students that there are alternative explanations 63Refers students to data as a resource for elaborating what they know 84

Encourages students to reflect on their observatory experience 92Evaluate Builds critical and logical thinking skills 83

89 Allows the teacher to observe and assess students’ knowledge and skills 96Allows students to assess their own learning 84

Strengths of the “Post-visit suggestions” is its capabilities to Engage the learner (score of 84 on 100 point scale) and promote the learner’s own involvement to Explain concepts and to Evaluate the learning experience (scores of 81 and 89). The product also has moderate capabilities in the Explore and Elaborate phases of the 5E model for promoting inquiry as a process for learning (scores of 73 and 72).

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 17 of 19

Page 18: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Pre-visit sun assessmentTotal 5E score: 69 on 100 point scale

Creates interest 88Engage Generates curiosity 84

88 Asks questions 88Elicits responses from students that uncovers what they know and think 92

Provides opportunities for students to describe objects and events 84Explore Encourages students to ask questions 71

59 Presents problem(s) for students to solve 54Encourages students to work together 42Uses contact with instruments to connect science and technology 46

Uses students’ previous experiences as a basis for explaining concepts 75Explain Allows students to connect astronomy to their school’s science

curriculum53

70 Encourages students to explain concepts in their own terms 92Asks students for justification or evidence to support their explanations 58

Encourages students to communicate their explanations to others 84Elaborate Encourages students to test their explanations 50

60 Reminds students that there are alternative explanations 54Refers students to data as a resource for elaborating what they know 50

Encourages student to reflect on their observatory experience 50Evaluate Builds critical and logical thinking skills 58

70 Allows the teacher to observe and assess students’ knowledge and skills 100Allows students to assess their own learning 71

Strength of the “Pre-visit sun assessment” is its capabilities to Engage the learner (score of 88 on the 100 point scale), specifically, its capability for eliciting responses from students that uncovers what they know and think (score of 92 on the 100 point scale). The project team also indicated the product has moderate capabilities for promoting the learner’s involvement in the Explain and Evaluate phases of inquiry-based learning.

Areas where the product could be strengthened or where it needs support of other products or components of the “Observing the Sun at McDonald Observatory” project are the Explore and Elaborate phases of the 5E model for promoting inquiry as a process for learning.

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 18 of 19

Page 19: Observing the Sun at McDonald Observatoryoutreach.as.utexas.edu/Marykay/McD_0105/final-evaluati…  · Web viewObserving the Sun at McDonald Observatory. Executive Summary. Background:

“Observing the Sun at McDonald Observatory”Formative Feedback Report

Evaluation of extent to which Product meets project objectives and requirements

Members of the project team provided formative feedback by completing a brief assessment of the extent to which the product is consistent with the 5E model for promoting student inquiry as a process for learning. The 5Es are Engage, Explore, Explain, Elaborate, and Evaluate. Items highlighted are paraphrase of statements of the project objectives.

Product: Viewing the sun with solar telescopeTotal 5E score: 72 on 100 point scale

Creates interest 96Engage Generates curiosity 92

87 Asks questions 84Elicits responses from students that uncovers what they know and think 75

Provides opportunities for students to describe objects and events 88Explore Encourages students to ask questions 84

74 Presents problem(s) for students to solve 58Encourages students to work together 46Uses contact with instruments to connect science and technology 96

Uses students’ previous experiences as a basis for explaining concepts 84Explain Allows students to connect astronomy to their school’s science

curriculum71

73 Encourages students to explain concepts in their own terms 79Asks students for justification or evidence to support their explanations 58

Encourages students to communicate their explanations to others 58Elaborate Encourages students to test their explanations 42

51 Reminds students that there are alternative explanations 50Refers students to data as a resource for elaborating what they know 54

Encourages student to reflect on their observatory experience 63Evaluate Builds critical and logical thinking skills 79

74 Allows the teacher to observe and assess students’ knowledge and skills 79Allows students to assess their own learning 75

Strength of the “Viewing the sun with solar telescope” is its capabilities to Engage the learner (score of 87 on the 100 point scale). The project team also indicated the product has moderate capabilities for promoting the learner’s involvement in the Explore, Explain, and Evaluate phases of inquiry-based learning (scores of 74, 73, and 74).

Elaborate (score of 51) is a phase of the 5E model where the product needs support of other products or components of the “Observing the Sun at McDonald Observatory.”

Evaluation Report – Third Coast R&D, Inc - 3/31/07 – Observing the Sun – page 19 of 19