Richard Jones

EdTech 505

Week 4 Assignment

Evaluation Reports

My Evaluation Topic

I am contemplating evaluating a product, called Cyber High, that is utilized during our school

day via classes called “Studies.” Our school utilizes this product as a way of offering credit

recovery for students who are at risk of not graduating and as a way for students to take college-

prep courses not offered on our campus. I am thinking that this will require an evaluation of the

product as well as the structure of its implementation in our school (the Studies classes).

Part 3A

1. Do I need to write a final evaluation report?

In general, it is important to provide an evaluation report (written or verbal) because it is the

medium through which the results of the evaluation are shared with the individual(s) who

requested it in the first place. Boulmetis and Dutwin (2011) make this clear in their opening

sentence when they state, “All the efforts you have expended up to this point in designing and

conducting the evaluation are of little consequence if you do not communicate the findings to the

people who need it” (p. 187). It is possible that the nature of the report will be based upon the

funding source of the program that is being evaluated.

In my case, a written evaluation report is needed despite not having a requested by the

individuals in charge of the program. I say this because implementation of the program

consumes a sizeable amount of the school budget, restricts access to the computer lab during

school hours, and reduces the number of traditional courses that are offered within the master

schedule. A program with that much impact should be evaluated; moreover, that evaluation

should be written and presented to all stakeholders as a way determining its effectiveness in

achieving the stated goals of the program. My report will be written and formal in nature.

2. For whom do I write the evaluation report?

The question of whom the evaluation is being written for can vary depending upon the funding

source and whether or not it is being written for a project manager. Boulemetis and Dutwin

(2011) suggest that the evaluator write the report for the person (program manager or other

funding source) who hired that evaluator. This does not mean that the hiring agency or project

manager will be the only audience for the report as this can vary.

In my case, I would be writing the report for our site principal, teachers, and some district

personnel. The principal will be the primary person for whom I write the report as he determines

whether or not to purchase and implement this courseware in our school. The teachers would

also be an audience as they might be responsible for implementing recommendations for

improving its effectiveness. Finally, district personnel would likely need to see the report as they

may use it as a way of justifying its expansion to other sites (should that be suggested in the

report). Thus, the principal is the primary individual for whom I will write this report as he is the

key stakeholder in funding the project; however, the evaluation findings will be important to a

number of other individuals.

3. What are the key components of the evaluation report?

There are several key components of an evaluation report. I have summarized the components

from pages 193-197 and placed them in the template that is provided in Exhibit 9.1 in the

Boulmetis and Dutwin text.

I. Cover Page

-Name of the evaluator and evaluating company

-Name of the project being evaluated

-Point at which the report is being released

II. Executive Summary

-Section One: Summary

-Purpose of the evaluation

-History of any previous evaluations

-List of conclusions and recommendations from the report

III. Table of Contents

-List of chapters in the report

-Appendixes in the report

IV. Introduction

-Section Two: Purpose of the Evaluation

-Describes aims of the evaluation

-Section Three: Background Info

-Provide information about foundation of program

-Tell what it was supposed to accomplish

-Describe evaluator’s role in the project

V. Methodology

-Section Four: Description of Evaluation Study

-Describe each question’s design

-Tell how information was obtained

-Describe the evaluation model being used by evaluator

-Describe methods of data collection and analysis

-List of participants involved

VI. Results and Findings

-Section Five: Results

-Include hard and soft data where appropriate

-Unexpected results

-Tables, graphs, and testing data

-Section Six: Program and the Results

-Interpretation of the results

VII. Summary and Recommendations

-Section Seven: Conclusions and recommendations

-Conclusions and recommendations by the evaluator

-Long-term

-Short-term

-Note emphasis on those that are more important

VIII. Appendixes

-Include sources and copies of any sources of data

-Charts that might have been too big for sections

Part 3C

Discuss Appendices B and C in general and/or specific terms.

Appendix B

Appendix B provides a sample evaluation report based on the Zoo in the Community project.

One of the initial aspects that stood out is the brief, but effective, description of the project’s

history, funding source, and goals of the funding. I was able to understand this without having

been associated with the project before reading this report. As I moved further into the report, I

would like to have seen the 8 objectives listed in the introduction section as opposed to waiting

until the one labeled “Evaluation Design.” However, this might not be necessary given the fact

that it was written for an individual who was familiar with the project.

There were many helpful components that I will look to utilize in my own report, especially as it

relates to data collection and reporting. First, the idea of a participant questionnaire is something

that I will likely need to implement in my own project as it will be important to get feedback

from the consumers of the program (students) as well as the people who implement it (teachers).

I appreciated how the report provided a brief statement about the purpose of each appendix item.

The reporting of the results was also impressive as the evaluator took the time to discuss the data

as it related to each objective.

The final aspect that stands out in this evaluation report is the fact that the evaluator provided 11

recommendations for the program despite having successfully met all eight of its goals. This

elevates the usefulness of the evaluation report from one that simply confirms success to one that

advances the program to another level. This would be especially important in addressing any of

the unexpected findings that might have surfaced as a result of the evaluation.

Appendix C

Appendix C provides a sample of an evaluation contract that a business might write up when

entering into an agreement with a client. The authors provide a list of tasks that will be

completed, the questions that will be used during the evaluation, and an overall plan for the

project (including a proposed budget). My initial reaction to this contract is that it is fairly

general in nature, meaning there is not much specific detail about the evaluation. The second

major reaction that I had to this piece was one of concern because the results sounded pre-

determined. In other words, it sounds like the company has already determined the need for

training without having completed an evaluation to determine the need for training. I guess a bit

more context would have helped in answering these two concerns.

I also had a couple questions about the budget that was constructed for this evaluation contract.

First, I noticed that it says “Year One.” Is this to assume that the contract would have to be

rewritten for subsequent years? Secondly, the company has made calculation errors in the first

two rows of the budget. Finally, I would have requested that the company define “fringe”

benefits had I been the purchasing organization.

Questions and concerns aside, this Appendix item provides an interesting look at how one would

go about constructing a contract for evaluation services. It gives a different view of the

evaluation process beyond describing the steps involved in carrying out the evaluation and

writing the report.

Bonus Activities

Option 1: Website Link

http://betterevaluation.org/evaluation-options/contractual_agreement

This website provides a basic definition of contract evaluation and gives a brief overview of the

components that will likely need to be included in one. This website also contains a link another

website where readers can open a contract evaluation template.

Option 2: HOTS for Appendix C

The assignment listed below was designed for use with Chapter Nine in the course text. This

assignment is meant to be completed in pairs. Here are the specifics:

1. Each student is required to write a 1-2 page scenario like the one listed on pages 185-186.

They will need to provide enough information that the reader would be able to answer

questions 1-3 on page 186. This scenario will be written on a Google Doc.

2. Students will need to share their scenarios with one other student (as assigned by the

instructor).

3. Students will answer questions 1-3 on page 186 based on the scenario that was created by their

partners.

4. Students will finish this assignment by getting together with their partners and sharing their

answers to the questions. The original writers will have an opportunity to discuss their

logic and how they envisioned the questions being answered for their scenario.