document resume author witta, e. lea; mathews, bruce … · document resume. ed 419 541 ir 057 043....

DOCUMENT RESUME

ED 419 541 IR 057 043

AUTHOR Witta, E. Lea; Mathews, BruceTITLE Comparison of 1995-1996 High School Students Attitude Survey

Results.PUB DATE 1997-00-00NOTE 47p.; Paper presented at the Annual Conference of the

American Evaluation Association (San Diego, CA, November4-8, 1997).

PUB TYPE Reports Research (143) Speeches/Meeting Papers (150)EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS Change; *Computer Assisted Instruction; *High School

Students; High Schools; Instructional Effectiveness;Instructional Improvement; Instructional MaterialEvaluation; *Interactive Video; *Student Attitudes; StudentReaction; Surveys

ABSTRACTPrior evaluation has been concerned with the attitude of

high school students toward the educational interactive video program for theyear in which they were participants. This analysis assessed whether therehad been changes in attitude toward the program from one year (1995) to thenext (1996) and if improvements made based on the 1995 evaluation had beeneffective. Three hundred forty-four (106 from 1995) high school studentsenrolled in an interactive video class participated in this evaluation.Change was assessed on seven constructs and by open-ended questionsconcerning strengths, weaknesses, and suggested improvements. This analysisrevealed that there has been a significant improvement in equipment and itsmaintenance. Strengths of the interactive video program have remainedrelatively constant over this two-year period. There were no significantdifferences in students' evaluation of the interactive video program over thetwo years, but there was concern over scheduling and behavior weaknesses.Several charts illustrate results. (AEF)

********************************************************************************* Reproductions supplied by EDRS are the best that can be made *

* from the original document. *

********************************************************************************

Comparison of 1995 - 1996 High School Students

Attitude Survey Results

E. Lea Witta

The University of Southern Mississippi

Bruce Mathews

Southwest Virginia Education & Training Network

Paper presented at the annual conference of the American Evaluation Association. San Diego,November 1997.

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

2

"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

E. Lea Witta

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)."

Comparison 95-962

Abstract

Prior evaluation has been concerned with the attitude of high school students toward the

educational interactive video program for the year in which they were participants. This analysis

assessed whether there had been changes in attitude toward the program from one year (1995) to

the next (1996) and if improvements made based on the 1995 evaluation had been effective.

Three hundred forty four (106 from 1995) high school students participated in this evaluation.

Change was assessed on seven constructs and by open-ended questions concerning strengths,

weaknesses, and suggested improvements. This initiates a trend analysis to be continued yearly.

3

Comparison 95-963

Prior evaluation with the data used in this study has been concerned with the attitude of

high school students toward the interactive video program (ITV) for the year in which they were

participants. The purpose of this analysis is to determine if there have been changes in attitude

toward the program from one year (1995) to the next (1996). This is the beginning of a trend

analysis to be continued each year. By comparing attitude changes from year to year, the impact

of improvements and changes to the program/facilities can be detected. If a supposed

improvement has had a substantial effect, it is hoped that effect will be detected. If the effect

reflects improvement in attitude, then it will be assumed the improvement caused this change.

This would suggest continuation of the improvement. If the effect seems to be detrimental, this

would suggest the problem should be approached by another method. In addition, this approach

permits facility administrators to anticipate or detect problem areas while they are still in a

manageable condition.

Subjects

All students enrolled in an interactive video class at this facility during the Spring semester,

1995 and Spring semester, 1996 were surveyed. Surveys were administered during the regularly

scheduled class time by the class instructor or remote facilitator. For analysis, respondents were

categorized by site (home or remote), by year of program existence (year 1 or year 2), and by

survey year (1995 or 1996). Of the 184 returned student surveys during Spring 1995, 106 were

completed by high school students. Twenty-one of the 66 home site respondents were

participating in a first year program. Fifteen of the 40 remote site respondents were participating

in a first year program (see Figure 1). Surveys were returned from respondents at nine different

schools in nine subject areas (see Figure 2).

4

Figu

re 1

Rem

Y2

(25

)

Rem

Y1

(15

Rem

Y1

(47

)

5

Res

pond

ents

By

Site

and

Yea

r19

95 N

=10

6

Hom

eY1

(21

)

Rem

Y2

1996

N=

238

Hom

eY1

(78

)

Hom

eY2

(45

)

Hom

eY2

(83

)

6

Figu

re 2 >,60

(,c7

;50

g40

11-

30a1 ?

20T

2.1

0c LL

I0

7

Cla

sses

Rep

rese

nted

by Y

ear

(199

5/19

96)

Ana

tom

y

Fin

Mgm

tC

reat

Wr

Spa

nish

Phy

sics

Eco

nom

ics

Eng

lish

SA

T

Yea

r Cal

culu

s

Cur

rent

Af

Fre

nch

Stu

dyS

K

Com

pute

r

Psy

chol

Latin

8

Comparison 95-966

There were 238 returned surveys by high school students during Spring 1996. Seventy-

eight of the 161 home site respondents were in a first year program. Forty-seven of the 77

remote site respondents were in a first year program. Surveys were returned for 12 subject areas

(all levels of Spanish were coded Spanish) from 14 high schools.

All student surveys (344) were entered for analysis. Three hundred nine responses were

coded as non-applicable. This was less than 5% (7392 responses). It was assumed that those

who marked non-applicable could not be ranked as undecided since that option was offered and

was not chosen. Since any numeric value assigned would bias the results (1=strongly agree, ergo

0 would be very strongly agree) and the proportion was relatively small, these were used as

missing values. In addition, 91 responses were not marked. These were also used as missing

values yielding a grand total of 400 missing values (<6%). Although the proportion of missing

values is relatively small, if listwise deletion were used many cases would be excluded from

analysis. To prevent this, mean substitution was used for factor analysis.

Measurement and Analysis

The survey instrument consisted of demographic information (school, gender, grade, etc.)

and 34 questions. Question 1 requested reason for taking the ITV class. Questions 2-31 were 5-

point Likert style questions. Questions 32-34 were open-ended requesting strengths, weaknesses,

and suggestions for improvement for the ITV program. Questions 21-26 were to be answered by

remote site participants only. Six questions were negatively stated in the questionnaire. These

were reverse coded for this analysis. Previous exploratory analysis has provided a seven-factor

model for the questionnaire as well as reliabilities of the factors for the Likert type questions in

common for both remote and home site students. While these models (1995, 1996) were

9

Comparison 95-967

similar, there were some discrepancies between them This analysis uses one model to conduct

multivariate analyses.

Responses to open-ended questions (Q32-34) were initially coded based on the response.

After determining similarities of the responses, these were placed in categories. For example, the

suggested improvements response "have schools on the same schedule" was coded as '38'. It and

the responses "synchronize time schedule" and "establish snow schedule" were then summed to a

major category "scheduling'. If a specific response was infrequent (<10) and could not

reasonably be included within a category, it was included in other. For this analysis, if either

survey year (1995, 1996) had sufficient responses to create a category, that category was included

for contrast even though one year may have zero responses. (Raw responses and their codes are

listed in Appendix A, Tables 1-3). Categories were then summed to major categories and the

proportion of categorical responses was contrasted by survey year (1995-1996) for all categories.

The assumption was made that there would be similar proportions of responses for each category

for each year. It must be remembered that these are not independent responses. A respondent

could submit three responses for strengths. All three responses may be variations of items coded

as Availability.

10

Comparison 95-968

Results

Strengths

There were 142 responses concerning strengths (106 respondents) in 1995 and 352 (238

respondents) in 1996. The proportion of responses citing classes offered at remote sites as a

strength had decreased (8% vs 1%) from 1995 to 1996. When, however, categories were

summed (ie, remote and availability became access - meeting people and open-minded became

interaction - technology and instruction became learning), no practical differences were detected

(see Figure 3). For example, in 1995, 39% of the respondents cited interaction as a strength. In

1996, 38% of the respondents cited interaction as a strength. This finding indicates that those

areas perceived as strengths in 1995 are still perceived as strengths in 1996.

These categories are further depicted by site (Figure 4) and by existence year (Figure 5).

Typically remote site students perceive availability or remote as more important (assuming

number of responses reflects importance) than home site students, while home site students

perceive meeting people or interaction aspects as more important than remote site respondents

(see Figure 4). When contrasted by year of program existence, students in the first year of the

program in 1995 did not differ from their counterparts in 1996 nor did those in the second year

(see Figure 5).

Figu

re 3

Acc

ess

(28.

17%

)

Str

engt

hs:

1995

/199

6H

igh

Sch

ool S

tude

nts

Oth

er (

2.11

%)

Acc

ess

(22.

16%

)

Oth

er (

5.97

%)

12

1995

N=

142

Beh

avio

r (5

.63%

)

Lear

ning

(24

.65

%)-J

Beh

avio

r (5

.11%

)

1996

N=

352

Lear

ning

(28

.69%

)

Inte

ract

ion

(39.

44%

)

Inte

ract

ion

(38.

07%

)

13

Figu

re 4

Beh

avio

r (7

.45%

)

Ava

il (1

1.70

%)

emot

e (7

.45%

)

Str

engt

hs:

1995

/199

6S

ite b

y S

urve

y Y

ear

Hom

e'95

Oth

er (

2.13

%)-

Inst

ruct

(17

.02%

)

Beh

avio

r (4

.56%

)

Ava

il (1

7.43

%)

emot

e (0

.00%

)

Oth

er (

6.22

%)

(Ava

il (3

5.42

%)

Mee

tPeo

p (4

2.55

Ope

nmin

(2.

13%

)T

echn

o (9

.57%

)

Hom

e'96

Inst

ruct

(14

.52%

)

1 4

Tec

hno

(13.

69%

)

Rem

ote'

95

Beh

avio

r

(2.0

8%)1

Rem

ote

(10.

42%

)O

ther

(2.

08 %

)

Beh

avio

r (6

.31%

)

Mee

tPeo

p (3

9.83

°Aai

l(3

0.63

%)

Ope

nmin

(3.

73%

)

Rem

ote'

96

Rem

ote

(1.8

0%)

Oth

er (

3.60

%)-

--d

Inst

ruct

(12

.61%

)

Mee

tPeo

p (2

9.17

%)

Ope

nmin

(0.

00

Tec

hno

(14.

58%

)

Inst

ruct

(6.

25%

)

Mee

tPeo

p (2

6.13

%)

--O

penm

in (

0.00

Tec

hno

(18.

92%

)

15

Figu

re 5

Str

engt

hs: 1

995/

1996

Pro

gram

Yea

r by

Sur

vey

Yea

rY

earl'

95

Beh

avio

r (1

.96%

)A

vail

(13

73%

)

mot

e (1

7.65

%)-

-

Oth

er (

0.00

%)-

Inst

ruct

(11

.76%

)

Beh

avio

r (5

.82%

)

Ava

il (2

4.34

%)-

Yea

r2'9

5

Beh

avio

r (7

.69%

) ID_

Mee

tPeo

p (3

7.25

%)k

vail

(23.

08%

)-

"-O

penm

in (

0.00

%)

Tec

hno

(17.

65%

)

1101

Rem

ote

(3.3

0%)-

---

Oth

er (

3.30

%)-

'

Mee

tPeo

p (3

8.4

-\41

Ope

nmin

(2.

20%

)In

stru

ct (

14.2

9%)

Tec

hno

(7.6

9%)

Yea

r1'9

6Y

ear2

'96

Rem

ote

(0.0

0%)-

-11

1111

1

Oth

er (

5.29

%)-

-

Inst

ruct

(12

.70%

)-

Beh

avio

r (4

.29%

)

Ava

il (1

8.40

%)

Mee

tPeo

p (3

3.33

%)

Rem

ote

(1.2

3%)

Oth

er (

5.52

%)-

' Ope

nmin

(3.

17%

)

Tec

hno

(15.

34%

)

Inst

ruct

(15

.34%

)-

Mee

tPeo

p (3

8.0

Ope

nmin

(1.

84%

)T

echn

o (1

5.34

%)

1617

Comparison 95-9612

Weaknesses

There were 128 responses to the weakness question in 1995 and 323 in 1996. The

proportion of responses citing audio problems had decreased from 35% in 1995 to 16% in 1996.

The proportion citing video problems had also been reduced from 19% to 2%. Minor equipment

problems, on the other hand, were not a category in 1995, but were 16% of the responses in 1996.

The scheduling category had also increased from 2% in 1995 to 12% in 1996.

When categories were summed (camera, sound, severe equipment problems, and minor

equipment problems formed equipment - remote and instruction formed instruction) differences

between 1995 and 1996 were still detected. In 1995, 61% of the weakness responses concerned

equipment failure or problems. This was reduced to 43% in 1996. The scheduling response

remained an increase in proportion from 1995 to 1996 (see Figure 6). These responses are

further depicted by site (see Figure 7) and by year of program existence (see Figure 8). Home

site responses (Figure 7) and responses by students in an established program (Figure 8) cite

behavior (cheating, lack of discipline) as weaknesses of the program more frequently than remote

site or those participating in a new program.

These results indicate that although there may still be equipment problems that need to be

resolved, there has been a significant improvement. More troubling is the increased incidence of

scheduling problems and the non-significant increase in behavior cited as a weakness. While a

proportional decrease in responses related to equipment requires an increase in another category

(proportions must sum to 1), 62 weakness responses were related to behavior (cheating, lack of

discipline, see Table A-2) in 1996 compared to 15 in 1995. In addition, 39 weakness responses

referred to scheduling problems in 1996 but only 2 in 1995.

18

Figu

re 6

Wea

knes

ses:

199

5/19

96

Inst

ruct

ion

(25.

00%

)

Oth

er (

0.78

%)

1995

N=

128

Sch

edul

ing

(1.5

6%)

r-B

ehav

ior

(11.

72%

)

Sch

edul

ing

(12.

07%

)

Inst

ruct

ion

(22.

60%

)

Oth

er (

3.41

%)

19

1996

N=

323

\--E

quip

men

t (60

.94%

)

Beh

avio

r (1

9.20

%)

- -

Equ

ipm

ent (

42.7

2%)

20

Figu

re 7

Sch

edul

ing

(2.7

4%)

Rem

ote

(2.7

4%)

Inst

ruct

(15

.07%

)-\

Oth

er (

1.37

%)-

-\ui

pMin

(0.

00%

)-

ipm

ent (

8.22

%)-

Cam

era

(23.

29%

)

Sch

edul

ing

(11.

48%

) -

Rem

ote

(7.1

8%)

Inst

ruct

(6.

70%

) -

Oth

er (

2.39

%)-

Equ

ip M

in (

20.5

7%)

Wea

knes

s: 1

995/

1996

Site

by

Sur

vey

Yea

rH

ome'

95

Beh

avio

r (9

.59%

)

Hom

e'96

Rem

ote

(10.

91%

)

Inst

ruct

(23.

64%

)--7

Sou

nd (

36.9

9%)

Beh

avio

r (2

6.32

%)

Sou

nd (

14.8

3%)

Cam

era

(2.3

9%)

Equ

ipm

ent (

8.13

%)

Rem

ote'

95 rSch

edul

ing

(0.0

0%)

Beh

avio

r (1

4.55

%)

Oth

er (

0.00

%)

Equ

ip M

in (

0.00

%)

Equ

ipm

ent (

5.45

%)

Cam

era

(12.

73%

)

Sch

edul

ing

(12.

39%

)

Rem

ote

(19.

47%

)

Inst

ruct

(19

.47%

)-

Rem

ote'

96

Sou

nd (

32.7

3%)

Beh

avio

r (6

.19%

)

Sou

nd (

18.5

8%)

Cam

era

(2.6

5

Equ

ipm

ent (

7.08

Equ

ip M

in (

8.85

%)

Oth

er (

5.31

%)

2122

Figu

re 8

Wea

knes

s: 1

995/

1996

Pro

gram

Yea

r by

Sur

vey

Yea

rY

earl'

95S

ched

ulin

g (1

.96%

)R

emot

e (0

.00%

)In

stru

ct (

5.88

%)

Oth

er (

0.00

%)

Equ

ip M

in (

0 00

%)

Equ

ipm

ent (

7.84

%)-

amer

a (2

7.45

%)

- -

Yea

rl '9

6

Beh

avio

r (1

1.76

%)

Sou

nd (

45.1

0%)

Sch

edul

ing

(9.2

7%)

Beh

avio

r (1

1.26

%)

Rem

ote

(15.

89%

)

Inst

ruct

(14

.57%

)

23

Oth

er (

5.30

%)

Yea

r2'

95

Sch

edul

ing

(1.3

0%)

Rem

ote

(10.

39%

)

Inst

ruct

(27

.27%

)-

Oth

er (

1.30

%)

Equ

ipM

in (

0.00

%)

d

Equ

ipm

ent (

6.49

%)-

'

Sch

edul

ing

(14.

04%

)

Sou

nd (

15.2

3%)

Rem

ote

(7.6

0%)-

Inst

ruct

(8.

19%

)C

amer

a (3

.31%

)

Equ

ipm

ent (

5.30

%)

Oth

er (

1.75

%)-

I--

Equ

ipM

in (

19.8

7%)

Equ

ipM

in (

13.4

5%)-

Yea

r2'

96

Beh

avio

r (1

1.69

%)

-Sou

nd (

28.5

7%

Cam

era

(12.

99%

)

Beh

avio

r (2

6.32

%)

Sou

nd (

16.9

6%)

Cam

era

(1.7

5%)

Equ

ipm

ent (

9.94

%) 24

Comparison 95-9616

Suggested Improvements

There were 93 suggested improvements made by high school students in 1995 and 230

made in 1996. The larger proportion of suggested improvements in 1995 were concerned with

audio (20% vs 11%), camera (9% vs 1%), and remote (21% vs 12%) than in 1996. Suggested

improvements in scheduling (1% vs 12%) and general equipment (2% vs 14%), on the other

hand, had increased in 1996. When audio, camera, and equipment categories were summed to

equipment (see Figure 9), although a larger percentage of the responses occurred in 1995 (31%),

it was very similar to 1996 (26%). The nature of the suggested equipment responses, however,

had changed. In 1995, the most frequent equipment suggested improvements were 'fix audio'

and 'fix camera'. In 1996, suggested improvements centered around new equipment (ie,

chalkboards, individual microphones).

When summed, suggested improvements in remote and scheduling remained the same.

No other categories had changed significantly. The suggested improvements remote category

included responses in 1995 indicating more remote classes should be offered. Obviously, more

have been made available since that suggested improvement has decreased.

These responses are further depicted by site (Figure 10) and program existence year

(Figure 11). These responses and the weakness responses suggest that while all equipment

problems have not been resolved, there has significant improvement.

25

Figu

re 9

26

Sug

gest

ed Im

prov

emen

ts:

1995

-199

6

IA/V

ar (

29.0

9%)

Env

ironm

ent (

8.18

%)

Oth

er (

1.82

%)

1995

N=

93

Rem

ote

(20.

91%

)I

Env

ironn

ient

(6.

09%

)O

ther

(9.

13%

)

IAN

ar (

19.5

7%)

Rem

ote

(12.

17%

)

1996

N=

230

Equ

ipm

ent (

30.9

1%)

Sch

edul

ing

(0.9

1%)

earn

ing

(8.1

8%)

Equ

ipm

ent (

26.5

2%)

Sch

edul

ing

(12.

17%

)

Lear

ning

(14

.35%

)

27

Figu

re 1

0S

ugge

sted

Impr

ovem

ents

:19

95-1

996:

Site

by

Sur

vey

Yea

r

Env

ironm

ent (

5.08

%)

Oth

er (

1.69

%)

Var

iety

(10

.17%

)

ctio

n (1

5.25

%)

Hom

e'95

Rem

ote

(13.

56%

)In

stru

ctio

n (1

.69%

)D

isci

plin

e (0

.00%

)

Env

ironm

ent (

8.45

%)

Oth

er (

11.2

7%)

arie

ty (

10.5

6%)-

Inte

ract

ion

(9.8

6%)

28

Env

ironm

ent (

14.7

1%)

Oth

er (

2.94

%)

Var

iety

(2.

94%

)In

tera

ctio

n (0

.00%

)

Cam

era

(15.

25%

)

Equ

ipm

ent (

3.39

%)

Sch

edul

ing

(1.6

9%)

Cla

ss (

6.78

%)

Hom

e'96

Aud

io (

14.0

8%)

Cam

era

(1.4

1%)

Rem

ote

(4.2

3%)

Inst

ruct

ion

(7.7

5%)

1---

Equ

ipm

ent (

19.0

1%)

Rem

ot'9

5

Aud

io (

20.5

9%)

Cam

era

(2.9

4%E

quip

men

t (0.

0S

ched

ulin

g (0

.0C

lass

(2.

94%

)

Rem

ote

(44.

12%

)

Rem

ot'9

6

Env

ironm

ent (

2.27

%)

Oth

er (

5.68

%)

Var

iety

(7.

95%

)

Inte

ract

ion

(10.

23%

)

Sch

edul

ing

(5.6

3%)

Cla

ss (

2.82

%)

Dis

cipl

ine

(4.9

3%)

Rem

ote

(25.

00%

)-

b.,

-Dis

cipl

ine

(2.9

4In

stru

ctio

n (5

.88

Aud

io (

5.68

%)

/-C

amer

a (0

.00%

)E

quip

men

t (7.

95%

)

-Sch

edul

ing

(22.

Cla

ss (

3.41

%)

Dis

cipl

ine

(4.5

5%)

Inst

ruct

ion

(4.5

5%) 29

Figu

re 1

1S

ugge

sted

Impr

ovem

ents

:19

95-1

996:

Yea

r by

Pro

gram

Yea

r

Env

ironm

ent (

8.82

%)

Oth

er (

0.00

%)-

\V

arie

ty (

8.82

%)

erac

tion

(2.9

4%)

Yea

rl'95

emot

e (1

4.71

%)-

Inst

ruct

ion

(0.0

0%)

Dis

cipl

ine

(2.9

4%)

Cla

ss (

2.94

%)

Sch

edul

ing

(0.0

0%)

Equ

ipm

ent (

5.88

%)

Env

ironm

ent (

4.17

%)

Oth

er (

12.5

0%)

arie

ty (

14.5

8%)

Inte

ract

ion

(13.

54 %

) -

33

Yea

rl'96

Aud

io (

35.2

9%)

Cam

era

(17.

65%

)

Env

ironm

ent (

8.47

%)

Oth

er (

3.39

%)

Var

iety

(6.

78%

)

Inte

ract

ion

(13.

56%

)

Rem

ote

(30.

51%

)

- -A

udio

(5.

21%

)C

amer

a (0

.00%

)E

nviro

nmen

t (7.

52%

)E

quip

men

t (11

.46%

)O

ther

(6.

02%

)

Sch

edul

ing

(2.0

8%)

Var

iety

(6.

02 %

)-.

Cla

ss (

1.04

%)

Dis

cipl

ine

(3.1

3%)

Inte

ract

ion

(7.5

2%)

Inst

ruct

ion

(11.

401

0.t

m(6

.02%

)

Inst

ruct

ion

(3.0

1%)-

--'

Dis

cipl

ine

(6.0

2%)

Cla

ss (

4.51

%)

Rem

ote

(20.

83%

)

Yea

r2'9

5

Yea

r2'9

6

Aud

io (

16.9

5%)

Cam

era

(6.7

8%)

z-E

quip

men

t (0.

0S

ched

ulin

g (1

.6

Cla

ss (

6.78

%)

Dis

cipl

ine

(0.0

0%'-I

nstr

uctio

n (5

.08%

)

Aud

io (

15.0

4%)

-Equ

ipm

ent (

17.

Sch

edul

ing

(19.

55%

)

31

Comparison 95-9620

Factor Comparison

Factors did not load on the same questions consistently from 1995 to 1996. The Audio

and Environment factors loaded on the same questions, and Materials Support and ITV

evaluation factors loaded on similar questions (one question was added to each in 1996). There

were several discrepancies, however, in the Student Behavior, Class Evaluation, and Interaction

factors. When the 1996 data was forced to load by the 1995 model, reliabilities did not differ

appreciably between survey years. When the 1995 data was forced to load by the 1996 model,

reliabilities again did not differ appreciably between survey years. Some questions, however, did

not fit either model substantively. For this contrast questions were placed on the factor which

they appeared to fit logically. Reliability for both groups in this model was then determined (see

Table 1). Although the reliability of the student behavior factor could have been increased to

0.62 by combining it with the interaction factor, this was not done. Exploratory analyses from

both survey years have yielded a seven factor model. To combine two factors would alter that

model significantly. The questions would also suggest that a separate factor could be established

distinguishing teacher from class. This also was not done. The new model was an adaptation of

the two previous models with as little change as possible while still providing a logical fit. Since

this model fit reliability analyses almost as well as either of the models developed from the

survey year data and it provided a logical explanation of the factors, it was used to contrast the

survey years.

Initially the mean for each respondent on each factor was determined and imputed into

the data. Multivariate analysis of these factors was then conducted using survey year (1995-

1996),

32

Comparison 95-9621

site (remote, home), and year of program existence (yearl, year2) as independent contrasts. Due

to the relatively small sample size in 1995 (one group would only have 15 members if site and

year were contrasted together), two analyses were run: one using site with survey year and one

using year of program existence with survey year. The purpose of these analyses were primarily

to determine if there had been changes in the interactive video program as students perceived it

from 1995 to 1996. There were no significant main effects of survey year. Since this program

had been rated so highly in 1995, any significant difference by survey year would most likely

indicate a change for the worse (see Figure 12). There was, however, a statistically significant

interaction effect of survey year with year of program existence (F=3.79', df=6,276, p<.001). The

effect size was very small (0.08). Audio was the only significant contributor (F=5.75, df 1, 281,

p<.02). This interaction is depicted in Figure 13 . The range of values for audio varied from a

2.11 (agree) to a 2.52 (agree). Although statistical significance was detected, there appears to be

little practical importance.

In addition, there were main effects of both year of program existence and site. These are

are not reported in this study.

'All statistics are reported using the square root of the raw scores. Homogeneity ofvariance (BoxM) was not achieved using raw scores but was established with the square rootfunction. Means are reported as raw scores.

33

Comparison 95-9622

Table 4

Factor Model for Contrasting Survey Years 1995. 1996

Factor LoadingQuestionNumber Label

Materials Support .73 Q4 Returned WorkQ28 Talk to Teach as neededQ29 Class materials timelyQ30 See Materials on System

ITV Evaluation. .79 Q14 ITV Good Addition CurricQ15 R-Hestitate Take Anothr ITVQ16 Choice ITV ClassQ17 ITV Good Way Offer ClassQ18 Take Another ITVQ19 R-ITV More Difficult

Audio .78 Q27 Hear Students other sitesQ31 Hear Quest other Sites

Environment .55 Q2 Amt Desk SpaceQ3 Clear sight TV

Student Behavior .53 Q8 Behav better ITVQ11 Better Listener

Class Evaluation .67 Q5 R- Limit ITV GradeQ7 R Most Talk by HomesiteQ10 R More Cheating ITVQ12 Study same ITVQ13 Tchr Attn Same Home/RemotQ20 R-More Study/Prep ITV

Interaction .47 Q6 Know Stud Other SchlQ9 Meet Other Schl Stu mre ofte

Figu

re 1

2

5 -

4.5

-4

-

c3.

5 -

co.)

CD

-

2.5

-

2-

1.5

-

1-

35

Fac

tor

Mea

ns19

95/1

996

Mat

Sup

tC

las

Eva

l

Yea

r

ITV

Eva

lS

tu B

ehav

Env

iron

Inte

ract

ion

Aud

io

36

Figu

re 1

3In

tera

ctio

n E

ffect

: Aud

ioS

urve

y Y

ear

by P

rogr

am E

xist

ence

Surv

ey Y

ear

- Y

ear

1Y

ear

2

3738

Comparison 95-9625

Summary

Strengths of the interactive video program have remained relatively constant over this

two-year period. Interaction with others, access to previously unavailable classes, and learning

(including technological exposure were most frequently cite in both years.

Responses to weaknesses of the program have, however, changed indicating significant

improvement in equipment. This, however, has been replaced with increased citation of

scheduling and behavior weaknesses. The scheduling weakness is also reflected as an increase in

suggested improvements.

There were no significant differences in students evaluation of the interactive video

program between those participating in 1995 and those participating in 1996. A significant

interaction effect detected between survey year and year of program existence showed very little

practical importance. It was, therefore, concluded there was no evidence of change in attitude

toward or evaluation of the interactive video program by high school students from 1995 to 1996

on this portion of the questionnaire. There are differences, however, in student evaluations by

site and by year of program existence.

Recommendations

This evaluation indicated there has been a significant improvement in equipment and its

maintenance. There are, however, other problems that have occurred. Due to the increased

frequency of response concerning problems with school schedules, this needs to be investigated

to determine if this problem can be resolved. Students need to be on similar schedules if at all

possible. In addition, there may be some problem developing in student behavior. Home site

students have indicated a lack of discipline and/or cheating at the remote site. This may or may

Comparison 95-9626

not become a problem, but does merit further investigation.

In addition, the questionnaire needs to be revised. Three factors (Class Evaluation,

Student Behavior, and Interaction) are not measuring consistently across years. This indicated

that the questions within these factors are not clear, or the factor needs to be strengthened with

additional questions. The cautions included concerning student behavior and class evalution my

also reflect this problem.

Appendix A

Table 1: Responses to Strengths

Table 2: Responses to Weaknesses

Table 3: Responses to Suggested Improvements

41

Comparison 95-9627

0

Comparison 95-96

Table 1

Codes for Answers given in response to the open-ended Strength

28

Factor Sub-factor Code Label

Interaction Meeting People 0 'Meet more people'2 'Interaction /Oth Schl'10 'Meeting different Schools'

Interaction Open-minded 21 'More Open-minded'30 'More Opinions'

Learning Technology 4 'Different - Rerun Lecture'6 'Picture'7 'Elmo'9 'Less intimidating'15 'Technology Exposure'17 'More Interesting'24 'Something New'25 'More I/A Cony'

Learning Instruction 5 'Instruction'19 'Better Understanding'20 'More Hands On'22 'Prepare for College'23 'Better Classes'26 'Quality Teacher'33 'Easier'34 'Question as needed'36 'More Learning'

Access Remote 1 'Remote'

Access Available Classes 3 'More can take'8 'Class Variety'11 'Wider Access'18 'Take Prey. Unavail Class'

Table 1 (Continued)

Comparison 95-9629


Behavior 12 'Greater Student Responsibility'13 'Less Cheating'14 'Increase Listening'29 'More Participation'35 'Better Study Habits'

Other 16 'None'27 'Saves Money'28 'More students/teacher'31 'Material Timely'32 'Better Environment'

Comparison 95-9630

Table 2

Weakness Response Codes By Factors and Factor Sub-divided


Behavior 0 'Increased Absence'4 'Easy to Cheat'5 'Noisy (Talking)'10 'Lack Participation'21 'Lack Discipline'29 'Pay less Attention'36 'Boring'

Equipment Sound 1 'Equipment - Sound'30 'High Voice not Heard'

Equipment Camera 2 'Equipment - Camera'14 'Video Intimidating'38 'Video Small'39 'TV Glare'

Equipment 6 'Time - Iniating Equipment'19 'Lack Communication'24 'Equip Fail - Severe'42 'Slow - Delays'

Equipment Minor 28 'Equip Fail - Occas'

Instruction 7 'Attention to Remote Students'8 '1-1 diff with remote'

12 'Teacher Attn to HomeSite'13 'Instruction Light'15 'Teacher attn divided'25 'Little Learning'35 'Difficult Learning'40 No Extra Help'

Table 2 (Continued)

Comparison 95-9631


Instruction Remote 9 'Teachers Need to be There'11 'Personal Contact Other Student'22 'No teacher at Remote'27 'Personal Contact Techer'31 'Impersonal - slight'34 'Remote less impt'

Scheduling 16 'Different School Holidays'23 'Schedule Conflicts'26 'Class Cancelled'

Other 3 '3 hour meeting too long'17 'Mail Late'18 'None'20 'Too many students/teacher'32 'Due date materials'33 'More space'37 'Large Classes'41 'No Field Trip'

45

Comparison 95-9632

Table 3

Suggested Improvement Response Codes By Factors and Factor Sub-divided


Equipment Sound 1 'Fix Sound'

Equipment Camera 2 'Fix Camera'

Equipment 16 'User-Friendly Equipment'17 'Facilitator initiate equipment pri'19 'Class on Equipment Use'29 'Fix equipment'30 'Trained Eqip/fix standby'42 'Need Chalkboard'47 'Better Communication'

Learning Class 3 'More class meetings - less lngth'14 'Require # Students/Site'21 'Reduce lecture time'23 'Longer Class Period'24 'Smaller Classes'41 'Rotate Home/remote'

Learning Discipline 8 'Better Discipline'28 'Better discipline - remote'44 'Screen students'

Learning Instruction 10 'No Learning'22 'Better Teacher Interaction'25 'Better Teachers'35 'Syllabus'37 'Experienced Teachers'38 'Remove facilitators'46 'Tutors'

Remote 15 'Facilitator at Remote'18 'Teacher needs to visit'

Table 3 (Continued)

Comparison 95-9633


Interaction 9 'More Interaction'13 'Some Total Class Meetings'20 'Invite More Schools'33 'More people'

Scheduling 26 'Synchronize time frame'27 'Same schedule'43 'Snowday Schedule'

Variety 4 'Advertise'5 'More locations'

12 'Offer More Classes'

Other 11 'Quit Using'31 'Internet use'32 'More exposure class content'34 'none'45 'Return work'

Environment 6 'Larger Monitor'7 'Better Setting'

36 'More seating'39 'Overhead Speakers'40 'Bigger Desks'

U.S. Department of EducationOffice of Educational Research and Improvement (OEM)

National Library of Education (NLE)Educational Resources Information Center (ERIC)

REPRODUCTION RELEASE(Specific Document)

I. DOCUMENT IDENTIFICATION:

ERIC

Title: Q,antioueltson A \C'n 1Clq le \N Nrk. e-VC/01 \-t,L41e-14

Author(s): , IADCorporate Source: INE1/4.42_,Noll uvr's On 49-9r

e 0

() its I° s-vr,

cam+ELP-c6.0swx Jc

Publication Date:

tr0

II. REPRODUCTION RELEASE:

In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in themonthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy,and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, ifreproduction release is granted, one of the following notices is affixed to the document

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottomof the page.

The sample sticker shown below will beaffixed to all Level 1 documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL HAS

BEEN GRANTED BY

\e

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

Level 1

Check here for Level 1 release, permitting reproductionand dissemination in microfiche or other ERIC archival

media (e.g., electronic) and paper copy.

The sample sticker shown below will be The sample sticker shown below will beaffixed to all Level 2A documents affixed to ell Level 29 documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL IN

MICROFICHE. AND IN ELECTRONIC MEDIAFOR ERIC COLLECTION SUBSCRIBERS ONLY,

HAS BEEN GRANTED BY


2A

Level 2A

ElCheck here for Level 2A release. permitting reproducgonand dissemination in microfiche and in electronic media

for ERIC archival collection subscribers only

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL IN

MICROFICHE ONLY HAS BEEN GRANTED BY

2B

\e


Level 28

Check here for Level 2B release, permittingreproduction and dissemination in microfiche only

Documents will be processed as indicated provided reproduction quality permits.If permission to reproduce Is granted, but no box is checked, documents will be processed at Level 1.

I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this documentas indicated above. Reproductith from the ERIC microfiche or electronic media by persons other than ERIC employees and its systemcontractors requires permission from the copyright holder. Exception is made for non-profit reproduction by Wanes and other service agenciesto satisfy information needs of educators in response to discrete inquiries.

Sign s'gn.lure

here,-* \17)2:'please °manizaummaress.

t--6 z .9cksu 4y jrIki1L-les

-__-.

A _

Printed Name/Position/Ms

. Lp,x_vi,-44 6311,

TelePtAll //) _)FAA:

E-Mai Address:

US in itZin°ate

(over,