final report blackjack 03-26-2004 this is the goooood one

33
Indiana University R561 Evaluation and C hange in the Instructional Development Process Orneal A. Brown, Alicia M. Clapp, Jung W. Hur, José M. Ochoa, Richter L. Tipton Spring 2004 Page 1 of 33

Upload: jose-manuel-ochoa

Post on 10-Apr-2018

219 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 1/33

Indiana University R561 Evaluation and Change in the Instructional Development ProcessOrneal A. Brown, Alicia M. Clapp, Jung W. Hur, José M. Ochoa, Richter L. Tipton

Spring 2004

Page 1 of 33

Page 2: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 2/33

Executive summary 3

I. Introduction 4

II. Methodology 52.1 The prototype 5

• Purpose• Strengths• Weaknesses• Features• Directions needed for instruction

2.2 Target audience 5• Demographics• Prior knowledge• Prerequisite skills• Attitudes

2.3 Tryout 6• Logistics (equipment, materials, human resources, time)• Environmental conditions• Observations

III. Results 7 3.1 Entry conditions (intended and observed) 8

3.2 Instruction (intended and observed) 93.3 Outcomes (intended and observed) 9• General observed outcomes• Outcomes by objective• Potential outcome correlations

3.4 Recommendations 16• About the prototype• About the target audience• About the objectives• About the tryout• About the instruments

IV. Summary 18

V. Appendices 20• Demographics• Pre-test and post-test• Reactionnaire• Observation• Job aid

Page 2 of 33

Page 3: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 3/33

To examine the instructional effectiveness of the video Winning Strategies:Blackjack, a round of user testing was conducted. After establishinginstructional objectives and designing the tryout experience, eightparticipants completed a demographics questionnaire, a pre-test, watchedthe video, practiced various hands of Blackjack, answered a post-test and areactionnaire. Among the main results obtained, the evaluator team foundthat 87% of the participants wanted to play Blackjack again, nobody went

backwards on their prior knowledge and everybody learned or mastered atleast one of the objectives. Due the nature of the tryout session and thenumber of players involved in the experience, the results are only applicable to this particular setting where we can conclude as well that this

video tape is somewhat effective in terms of allowing learners to be able toname the components of the Blackjack table, identify the correct definitionof each term, identify applicable basic rules, and use appropriate signals;this video is not effective in the light of teaching various Blackjack strategies, but in the perspective of allowing learners to express desire toparticipate in a Blackjack game, the video seems to be effective.

Page 3 of 33

Page 4: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 4/33

This evaluation was conducted on Goldhil Home Media’s 1999 training video Winning

Strategies: Blackjack , Universal Product Code (UPC) 743452007233. The video is 50

minutes long, hosted by James Coburn, and based on the strategies of Frank Scoblete. The

evaluation team consisted of five graduate students from Indiana University: Orneal Brown,

Alicia Clapp, Jung Won Hur, Jose Ochoa, and Richter Tipton.

The video does not include learning objectives. After repeated viewings of the material, the

evaluation team developed six objectives which seemed to be the purpose for the training.

These objectives were evaluated through a series of surveys, tests, and observations (see

Appendix A, B, C, and D). The evaluation period lasted 60 minutes and a detailed time

breakdown is listed in section 2.3.

The learners involved in this evaluation were eight volunteers who expressed a desire to learn

more about the game of Blackjack. All participants were novices or beginners with the most

experienced learner indicating they had played Blackjack ten times or less.

The training material was presented and evaluated differently from what the evaluation team

believes the designers intended. This report contains a complete discussion of these

differences as well as the potential problems as a result of these changes.

The evaluation team used a focus group format and a practice session in an attempt to

reinforce the skills presented in the video. The general methodology for the evaluation

included a demographic questionnaire and a pre-test.

The students were then shown a portion of the video. The learners participated in a 16 minute

practice session in a simulated environment during which time their actions were documented

by observers. The students then took a post-test and a reaction survey.

Page 4 of 33

Page 5: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 5/33

2.1 The prototype

The training program consists of a 1999 VHS video by Goldhil Home Media’s Winning

Strategies: Blackjack .

• Purpose: To train beginner and intermediate Blackjack players on the components of

a Blackjack table, terminology, applicable rules, hand gestures and strategies to beat the

dealer on a casino environment.

• Strengths: In order to arouse audience’s interests, the video uses fast bit backgroundmusic and shows casino gambling scenes in a very positive way. The video covers a lot of

information: from the basic rule, such as how to count the card value, to the more

sophisticated rules, such as what strategy a player should use in a specific situation. It

divides the content into several sections, and provides a summary after explaining several

definitions.

• Weaknesses: The pace of the video seems to be too fast, the amount of information

can be overwhelming, and no specific directions or instructions are provided to the

audience.

• Features: Rules of the Game, Hitting, Surrendering, Doubling Down, Money

Management, Playing in the Casino, Standing, Splitting Pairs, Insurance

• Directions needed for instruction

The video used is self-explanatory and self-paced.

2.2 Target audience

• Demographics: The target audience includes individuals (males and females) that

expect to play Blackjack in a casino setting. This is important because the tips, techniques,

and strategies are geared toward beating the house (dealer) and not to beating the people

at the table. The viewers are expected to be 21 years or older (the legal age for gambling).

Page 5 of 33

Page 6: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 6/33

• Prior knowledge: No previous knowledge of Blackjack is necessary. However, users

should be able to distinguish each card, such as an ace and a king. If they have some basic

knowledge about general card game, they can get more benefit from the video.

Prerequisite skills: Geared toward the beginning player but also serves as anintermediate player refresher course.

• Attitudes: Interested in casino style Blackjack, and probably have enough disposable

income to visit a casino and maybe lose their money. Positive attitude towards card games,

gambling, and casinos is expected.

2.3 Tryout.

• Logistics (equipment, materials, human resources, time)

Event Description Length

Welcome A brief introduction to the lesson. Get everyone seated,explain what people will be learning and describe the basicoutline for the session.

1 min

DemographicsSurvey

Hand out a one-page paper survey to each person tocomplete. Collect the surveys upon completion.

2 min

Pre-test Hand out a four page pre-test to each person. Explain thatthis is intended to assess their level of knowledge coming

into the session.

5 min

Watch the video

The video will be turned on and allowed to play for the firsttwo segments uninterrupted. As the video is playing, thegroup will observe and record behaviors exhibited by participants. Observers will look for cues about emotionalstate, when/if participants take notes, and interaction between participants (whispering, glances) etc.

25 min

Practical test Immediately following the video, participants will beinvited to play a few games of Blackjack. Two tables will beset up with 3-4 players and one dealer at each table. Thegame will be played as it would be in casino with multipledecks. The table will provide the same cues about the rulesof the game as a casino table would. There will be one

observer for each table.

16 min

Reactionnaire Hand out a four-page reactionnaire to each participant.Collect the reactionnaire upon completion.

5 min

Post-test Hand out a post-test to each participant, which is similar tothe pre-test taken before the instruction. Collect the post-test upon completion.

6 min

Page 6 of 33

Page 7: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 7/33

• Environmental conditions

The intended environment for viewing the video is a home living room with a TV and VCR.

However, the environment for our training is a classroom setting with VCR and a

television.

The evaluation portion of the training includes tables with dealers, cards, chips, and

seating for four learners.• Observations

Observers recorded the cards of each player and the actions taken in order to assess the

player’s application of the strategies provided. They also looked at whether or not players

follow the basic rules of the game while playing (i.e. not touching card, knowing when they

have won/lost, using proper cues to signal actins etc…)

Expected Actual Discrepancy

E n

t r y c o n

d i t i o n s

Learners are familiar with betting

(experience in a casino).

50% of the learner had noexperience of betting in acasino; 37% of the learners

had little experience, and13% of the learners hadsome experience.

Mostly all of the learners hadlittle to no experience bettingin an actual casino.

Novice or beginner(experience withBlackjack in a casino).

87% of the learners werenovice players of Blackjack in a casino; 13% were beginner players of Blackjack in a casino.

With this test group, the targetaudience was reached.

Interested in learning toplay Blackjack.

50% of the learners strongly agreed to having an interestin learning to play Blackjack and 50% of the learnersagreed to have an interest inlearning to play Blackjack.

All learners of this groupshowed positive interest inusing an instruction tool thattaught Blackjack.

Individual, at home,stop/rewind. Group, in class, no stop. Learners viewed the video in a

classroom situation.

Take notes. Nobody takes notes. Students were prevented fromreviewing material.

Page 7 of 33

Page 8: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 8/33

Expected Actual Discrepancy

I n s t r u c t i o n a l c o n

d i t i o n s

No practice. With practice. Students were allowed topractice the behaviors in asimulated environment.

Whole video. Only first 25 minutes of video.

The training experience lackedthe second 50% of the video.

O u

t c o m e s Objective 1: Given a

Blackjack table,learners will be able toname its components with 80% accuracy.

Three participants achieved80% accuracy or greater when asked to label adrawing of a Blackjack tableon a post-test.

Five participants did not meetmastery level requirements forobjective one.

Objective 2: Givenspecific Blackjack terms, learners will beable to identify thecorrect definition of each term with 80%accuracy.

Four participants achieved80% accuracy on post-testquestions relating toterminology.

Half of all participants did notmaster the terms associated with Blackjack.

Objective 3: Given aBlackjack table,learners will be able toidentify the 12applicable rules of Blackjack.

Seven participants achievedmastery of rule relatedquestions on a post-test.

One participant did notachieve mastery of theBlackjack rules.

Objective 4: Given aBlackjack playingenvironment, learners will use the correcthand signals to hit,stay or double down.

Four participants were ableto correctly identify themeaning of hand gestures100% of the time whenasked through fill-in-the blank questions on a post-test. When observed, threeparticipants used theappropriate hand gesture fortheir stated desired action100% of the time. All othersdid so 80% of the time ormore.

Half of the learners were notable to provide the correctsignals on a post-test.However, this result differedfrom the observed resultsduring play where allparticipants achieved at least80% mastery.

Objective 5: Afterseeing the player’scards and the dealer’sup-card, learners willtake correct action based on the strategiesprovided in the video.

All players were able to statethe appropriate action 50%of the time or less on a post-test. When observedplaying, three participantsfollowed through with thecorrect actions 80% of thetime. All others did so 60%of the time or less.

No players were able toachieve mastery of the playingstrategies on a written post-test. Five were not able to doso during actual play.

Page 8 of 33

Page 9: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 9/33

Expected Actual Discrepancy

Objective 6: After watching the video,learners will state

desire to participate ina game of 21 if asked.

83% of participants statedthat they were interested inplaying Blackjack again

when asked on areactionnaire. 50% indicatedinterest in playing Blackjack in a casino.

The objective was met forplaying Blackjack in general, but not for playing Blackjack in a casino.

3.1 Entry conditions (intended and observed)

The results of the demographic tool produced information that gave us an idea of what level of

expertise all of the learners considered themselves to be. Criteria such has experience level,numbers of times played and interest in the game of Blackjack, were used to determine the

learners general views of Blackjack, before the viewing of the video.

We first reviewed the results that gave us an indication of how many learners possibly knew

how to bet, in a casino. 50% of the learners selected No Experience, 37% of the learners

selected Little Experience, and 13% of the learners selected Some Experience, these were the

only terms that each of our learners chose to describe themselves.

The learners were also asked to discuss the number of times they had ever played Blackjack ina casino. Based on a scale of 0-5, 6-10, 11-15, 16-20, and 20 or more times, we found that the

results of this question were somewhat difficult to understand. 87% of our learners selected the

0-5 times as an option, and 13% stated they had practiced Blackjack in a casino 6-10 times.

0 - 5

t i m e s

( n o v i c e )

6 - 1 0 t i m e s

( b e g i n n e r )

1 1 - 1 5 t i m e s

( i n t e r m e d

i a t e )

1 5 - 2 0 t i m e s

( e x p e r i e n c e d

)

2 0 + t i m e s

( e x p e r t )

Player 1

Player 2

Player 3

Player 4

Player 5

Page 9 of 33

Page 10: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 10/33

Player 6

Player 7

Player 8

The level of interest in learning to play Blackjack was very high. 50% of our learners strongly

agreed that they were interested in learning, and the other 50% agreed that there were

interested in learning to play.

Strongly agree50%

trongly disagree50%

3.2 Instruction (intended and observed)

The Blackjack video used in this training was not implemented in the same situation our team

believes it was designed for. The video appears to have been developed to be used by an

individual who has the ability to stop and revisit the Blackjack strategies at will. We believe the

pace of the video is too fast to be understood in one sitting—this belief is supported by data

from the observations, post-test, and the reactionnaire. The video is also designed to be viewed

in its entirety.

Actual deployment of the video was conducted in a group setting and only the first 25 minutes

of the video were viewed. The 25 minute video presentation was presented continuously and

allowed for no revisiting of the material. To support the video, the team designed a practice

session where the learners were able to practice the strategies and concepts with their peers in

Page 10 of 33

Page 11: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 11/33

a risk free environment. A bullet point summary of the intended and actual deployment

methodologies is provided at the beginning of this section.

3.3 Outcomes (intended and observed)

3.3.1 General observed outcomes

Overall, the results of the post-test showed that as individuals, all participants’ learned from

the instruction. This is shown by the graph below which shows the average scores of the pre

and post tests by participant. All participants increased their scores with the smallest gain

being 11.1% (Player 4) and greatest gain being 36.1% (Player 1). Only Three participants,however, achieved overall mastery of greater than 80%. The class average increased from

54.2% to 76.4%. Though this does show marked improvement, the final class average falls

short of the 80% mastery level.

22.2

63.969.4

61.166.7

41.7

50

58.358.3

77.8

94.4

72.2

83.3

69.472.2

83.3

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8

Players

S c o

r e s

Pre-test Post-test

3.3.2 Outcomes by objective

The following graph shows that for each objective measured by the pre and post tests, the class

as a whole demonstrated an increased level of mastery. The largest improvement was shown

for the first objective, ability to label the components of the table, and the lowest level of

Page 11 of 33

Page 12: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 12/33

improvement was shown for the fifth objective, application of the strategies for playing

Blackjack.

10.1

54.7

72.566.7

40.6

56.25

78.1

90.8

79.2

46.9

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5

Objectives

S c o r e s

Pre-test average Post-test average

The following is a more detailed breakdown of the outcomes for each objective.

Objective 1: Given a Blackjack table, learners will be able to name its

components with 80% accuracy. As shown in the graph below, three participants

achieved 80% accuracy or greater when asked to label a drawing of a Blackjack table on a post-

test. Five participants did not meet mastery level requirements for objective one; however, all

participants did show improvement from the pre-test results to the post-test results. This

shows that some learning did occur for all participants.

Page 12 of 33

Page 13: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 13/33

0

13.716.7 16.7 16.7

0 0

16.716.7

66.7

100

33.3

50

83.3

16.7

83.3

0

1020

30

4050

60

70

8090

100

1 2 3 4 5 6 7 8

Players

S c o r e s

Pre-test Post-test

Objective 2: Given specific Blackjack terms, learners will be able to identify

the correct definition of each term with 80% accuracy. The table below shows that

four participants achieved 80% accuracy on post-test questions relating to terminology. Half of

all participants did not master the terms associated with Blackjack. One participant showed no

improvement in mastery level. Complete mastery (100%) was shown by two participants.

25

62.5

75

62.5 62.5

50

62.5

37.5

25

87.5

100

75

87.5

75

100

75

0

1020

30

4050

60

70

8090

100

1 2 3 4 5 6 7 8

Players

S c o r e s

Pre-test Post-test

Objective 3: Given a Blackjack table, learners will be able to identify the 12

applicable rules of 21. As can be seen in the graph below, seven participants achieved

mastery (80% correct) on rule related post-test questions. One participant, Player 6, failed to

achieve mastery of the Blackjack rules. Dramatic improvement was shown by Player 1.

Page 13 of 33

Page 14: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 14/33

26.7

8086.7

80

93.3

53.3

66.7

93.393.3

86.7

100

93.3

100

60

93.3

100

0

1020

30

4050

60

70

8090

100

1 2 3 4 5 6 7 8

Players

S c o r e s

Pre-test Post-test

Objective 4: Given a Blackjack playing environment, learners will use the

correct hand signals to hit, stay or double down. The following graph shows the

results of the post-test items for objective four. Four participants were able to correctly identify

the meaning of hand gestures 100% of the time when asked through fill-in-the blank questions

on the post-test. Half of the learners were not able to state the correct signals for given action.

33

100 100

67 67 67

33

6767

100 100

67

100 100

33

67

0

1020

30

4050

60

70

8090

100

1 2 3 4 5 6 7 8

Players

S c o r e s

Pre-test Post-test

The observation results shown in the next graph, however, indicate a much higher level of mastery than those of the post-test. When observed, three participants used the appropriate

hand gesture for their stated desired action 100% of the time. Though this was one fewer than

on the post-test, all other participants achieved 80% mastery or greater. However, this result

differed from the observed results during play where all participants achieved at least 80%

mastery.

Page 14 of 33

Page 15: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 15/33

2

0 0

1 1 1

0

5

8

7 7

8

6 6

8

3

0

1

2

3

4

5

6

7

8

9

1 2 3 4 5 6 7 8

Players

O b s e r v a

t i o n s

Incorrect gestures Correct gestures

Objective 5: After seeing the player’s cards and the dealer’s up-card, learners

will take correct action based on the strategies provided in the video. The graph

below shows that no participants reached the 80% mastery level for the strategy objective,

objective five. All players were able to state the appropriate action only 50% of the time or less

on a post-test. Furthermore, very little gain was shown overall between the pre-test and post-

test scores. This is also the only objective in which any participants showed a lower level of

proficiency on the post-test than that of the pre-test.

25

50 50 50 50

25

50

25

50

25

50 50 50 50 50 50

0

1020

30

4050

60

7080

90

100

1 2 3 4 5 6 7 8

Players

S c o r e s

Pre-test Post-test

As was the case with the fourth objective, the results of the observation were higher than those

shown by the scores of the post-test. The first graph below shows the average number of times

that the participants followed the strategies.

Page 15 of 33

Page 16: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 16/33

6

8 8

2

6

8

4

6

0

1

2

3

4

56

7

8

9

10

1 2 3 4 5 6 7 8

Players

A v e r a g e

t i m e s

f o l l o w e

d s

t r a t e g

i e s

The second graph shows the number of correct hands played and the number of incorrect

hands played. When observed playing, three participants followed through with the correct

actions for 80% of the hands played. All others did so 60% of the time or less. No players were

able to achieve mastery of the playing strategies on a written post-test. Five were not able to do

so during actual play.

1 1 1

4

2

1

3

2

3

4 4

1

3

4

2

3

0

1

2

3

4

5

1 2 3 4 5 6 7 8

Players

H a n

d s p

l a y e

d

Incorrect plays Correct plays

Objective 6: After watching the video, learners will state desire to participate

in a game of 21 if asked. 87% of participants stated that they were interested in playing

Blackjack again when asked on a reactionnaire. 50% indicated interest in playing Blackjack in a

Page 16 of 33

Page 17: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 17/33

casino. The objective was met for playing Blackjack in general, but not for playing Blackjack in

a casino.

3.3.3 Potential outcome correlations

Due to the small sample size of the learners (8 people), statistical analysis of the data is not

valid at this time; however, the data does show three interesting potential correlations that

should be compared in future evaluations.

First, a negative correlation appears to develop when you compare interest in playing Blackjack

in a casino against the belief that Blackjack is a game of luck (r= -0.577 indicates a weak

negative relationship).

Second, no correlation appears to develop when you compare interest in playing Blackjack

again (but not necessarily in a casino) against the belief that Blackjack is a game of luck (r= -

0.220 indicates no relationship).

Third, against the assessment team’s prediction, there appears to be a very weak negative

correlation developing between the learner’s desire to play again and the adequacy of the time

allotted for practice (r= -0.477indicates a very weak negative relationship).

3.4 Recommendations

Prototype : Based on our initial data, we recommend that this video be viewed completely.

We further suggest that the learners be allowed to review the strategy portions of this video

repeatedly. Learners should be encouraged to take notes throughout the video. To support

this recommendation, the evaluation team has included in the appendices a matrix whichincludes the strategies for hard, soft, and split pair hands. We believe the learners would

benefit from this matrix. The evaluation team also recommends continued implementation of

the practice session to support the internalization of the strategy presented in the video.

Target Audience : While our audience consisted largely of volunteers, we believe this video

was produced for individuals actively seeking Blackjack strategies for use in a casino

Page 17 of 33

Page 18: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 18/33

environment. For this reason, we recommend that future learners be recruited through a

system where they take an active part in seeking and signing up for the training.

Objectives : In retrospect, setting an achievement level where 80% of the learners

demonstrating an 80% proficiency in all learning objectives may have been high enough given

the instructional situation (materials, time, and interest). If this prototype methodology is to

be replicated, the evaluation team suggests changing the standards to show positive

improvement in all instructional objectives.

Tryout : Changes to the prototype methodology include providing the learners with a job aid,

recommending note taking in the beginning of the instruction, viewing the video completely,

and increasing the time and interaction of the practice session. Further, the evaluation team

believes the learners would benefit from using the strategy job aid during the practice session.

Instruments : After evaluating the data provided through our instruments, the evaluation

team recommends reducing the number of items in the reactionnaire and the demographics

instrument. Some of the information is redundant and of little value to evaluation. Further

reductions may be possible (even likely) after enough data is collected to evaluate the reliability

and validity of the data collected.

• Create an observation sheet that allows observers to easily follow the events of the

practice sessions, which includes having only one specific coding process and check

boxes next to predetermined outcomes.• State that there will only be a specific number of hands that will be played, this will allow

enough time for the learners to state their intended strategies aloud, and give the

observers enough time to record all of the data of the hands being played.• We recommend the use of more observers; we feel there should be 1 observer for every

two players. This will allow for a more accurate amount of data collected.• Future pre test and observations tool should be more closely linked. This will allow

evaluators to compare statements, made by the learners, early in the workshop to results

of how learners played once the video had been viewed.• We believe that the success criteria may have been somewhat high to properly judge the

actual success of the learners.

• We recommend some incentive for learners while playing Blackjack during the practices

session of the workshop.• We recommend further development of the strategy portion of the pre and post test.

Instead of giving learners the option of selecting hit as an answer, we recommend

Page 18 of 33

Page 19: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 19/33

splitting that option to read; "hit to 17", or "hit to 12". This will allow evaluator to see

whether learners were actually responding to the card strategically.• We would recommend that all questions that involves role playing, as either the player or

the dealer, should clearly express the current mindset the learner should have.• Instead of giving the learners the option to select only one answer, we should give them

the opportunity to select all answers that would best answer the question.• At the beginning of the pre and post test be sure the learners know the rules of the casino

so they are able to answer the questions more accurately.• We believe the reactionnaire, should require the learners to further discuss their

objection to a specific question or topic. This will produce more valid responses.

• During the demographic section of the workshop we recommend that the learners be

asked exactly how many times they have played Blackjack. This will allow evaluator to be

able to know exactly how many times users had played the game.• Further development of the demographic section of the workshop should include specific

questions that ask rather learners would be interested in learning strategy, or the rules of

Blackjack.

For design team: • One of the key results found in the workshop was that the learners felt as thought the pace

of the video was too fast. We would recommend the video to have a slower pace, so that

learner can have an opportunity to absorb the information, provided by the video.• Our team recommends that video should tell the learners that it would be best to take

notes while viewing the video. We believe it should suggest stopping and rewinding the

video, to further understand the more difficult lessons in the video. We also recommend

that the learners should be given a job that will allow them to follow along with the video.

• Be sure to show examples of success, so that the learner will know that the intended goal of

this video is to increase the success in winning Blackjack.• Our team believes that the interview participants should be modified.

Page 19 of 33

Page 20: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 20/33

The goal of this report is evaluating the video, “Winning Strategies: Blackjack” in terms of six

objectives that this video is intended to achieve. In order to evaluate this product, we invited

eight volunteers who were beginners or novice players and conducted a tryout using various

instruments. Participants first filled out the demographic sheet followed by the pre-test. They

then watched the video for 25 minutes without stopping and practiced the game in groups for

16 minutes. While they practiced the game two evaluators observed their behaviors. After

practicing it, they were asked to take the post- test and fill out the reactionnaire sheet. Notice

that due to the time limitation, only some portion (25 minutes) of video was shown. The

outcomes are as follows.

• All participants’ learned from the instruction. The class average increased from 54.2%

to 76.4%. Though this does show marked improvement, the final class average falls

short of the 80% mastery level.

• Three participants achieved 80% accuracy or greater when asked to name the

components of the Blackjack table (objective1). Although five participants did not meet

mastery level requirements, all participants did show improvement from the pre-test

results to the post-test results.• Four participants achieved 80% accuracy relating to terminology (objective2).

• Seven participants achieved mastery on applicable basic rules (objective 3).

• Four participants were able to correctly identify the meaning of hand gestures 100%

accuracy.(objective 4)• All participants achieved less than 50% accuracy relating to the taking the action

provided in the video. (objective 5)

• 87% of participants wanted to play the Blackjack again(objective 6)

Since the number of participants was small and the tryout conditions were different from what

the video originally intended, it is hard to generalize the results. However, based on what the

eight participants performed on the tryout we can make the following decisions.

• This video tape is somewhat effective in terms of allowing learners to be able to name

the components of the Blackjack table, identify the correct definition of each term,

identify applicable basic rules and use appropriate signals. However, due to the fast

speed of the video and the lack of previous experience, some participants could not

achieve the objectives. Therefore, this product should be developed more slowly and

provide more detailed information for beginners or novice players.

Page 20 of 33

Page 21: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 21/33

• This video is not effective in the light of teaching various Blackjack strategies. Almost

all participants could not learn what strategy they could apply in a certain situation. In

particular, we noticed even though a participant took a right action, he did it by

guessing not knowing the exact strategy. We came up with participants could not learn

necessary strategies since the video moves too fast, and there is no job aid or specific

examples that assist them in understanding. Consequently, this video should be

redesigned in a more instructive way, adding job aid and specific examples.

• In the view of allowing learners to express desire to participate in a Blackjack game,

the video seems to be effective. However, because all participants were volunteers who

would like to learn the game, we cannot say it is effective enough. Therefore, following-

up research, such as interviewing, is necessary.

A. Demographics

B. Pre-test and post-test

C. Reactionnaire

D. Observation

E. Job aid

Page 21 of 33

Page 22: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 22/33

D E M O G R A P H I C S

Name: Date:

Please rate your current experience level with each item using the scaleprovided.1

N o

e x p e r i e n c e

L i t t l e

e x p e r i e n c e

S o m e

e x p e r i e n c e

S i g n

i f i c a n

t

e x p e r i e n c e

E x

t e n s i v e

e x p e r i e n c e

1 General Card Playing

2 Blackjack

3 Casino Gambling

2 Please rate your comfort level in each area using the scale provided.

N o

t

c o m

f o r t a

b l e

S o m e w

h a

t

u n c o m

f o r t a

b l e

N e u

t r a

l

S o m e w

h a

t

c o m

f o r t a

b l e

C o m p

l e t e l y

c o m

f o r t a

b l e

4 General Card Playing

5 Blackjack

6 Casino Gambling

7 Basic Addition

Page 22 of 33

Page 23: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 23/33

Please indicate the number of times you have participated in the followingactivities:3

0 –

5 t i m e s

6 -

1 0 t i m e s

1 1 -

1 5 t i m

e s

1 6 –

2 0 t i m

e s

2 0 +

t i m e s

8 General Card Playing

9 Playing Blackjack (in a casino)

10 Playing Blackjack (elsewhere)

11 Casino Gambling

4 Please rate each item using the scale provided.

S t r o n g

l y

d i s a g r e e

D i s a g r e e

N e u

t r a

l

A g r e e

S t r o n g

l y

a g r e e

12 Blackjack is a game of luck.

13 Blackjack is a game of strategy.

14 I adhere to a particular strategy when playingBlackjack.

15 I am interested in learning to play Blackjack.

16 I am interested in learning how to win atBlackjack

T h a n k y o u !

Page 23 of 33

Page 24: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 24/33

Name: Date:

The table below shows final total value of a dealer and player. Check the box next to thewinning hand. 1

Dealer Player

1 21 23

2 15 18

3 19 17

In the table below, the figures in parentheses display cards that a dealer and a player held.Check the box next to the winning hand.2

Dealer Player

4 ( 9, 10) (4, 7 , 9)

5 (Ace, 10) (king, 8, 2)

6 (5, 7, king) (queen, 3, 5)

7 (queen, 9) ( 2, 6, 8)

3 Write down the number value of each card.

8. ( ___ or ___ ) 9. ( _____ ) 10. ( ___ or ___ ) 11. ( _____ )

Page 24 of 33

Page 25: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 25/33

4 Fill in the blanks with the appropriate term .

12 A player and dealer end up with the same total value, it is considered a ____________.

13 When your Ace can count as an 11 or 1, you are holding a ____________________ hand.

14 ________________ means getting an additional card.

15 ________________ means that a player does not want to get an additional card.

5 Write a “T” on the line if the statement is True , or an “F” if is False .

16 When a dealer gets his/her cards, both cards should be face up.

17 If a player wants to get an additional card, he/she should tap the table.

18 A dealer must get an additional card on any hand totaling 16 or less.

19 If a player wants to double the bet, he/she should place the additional bet ontop of the original bet.

20If a player busts, they automatically lose even if the dealer subsequentlybusts.

21 Only the time when a player splits the hands, he/she can touch the card.

22 A player must focus on a dealer’s “hole card”.

6 Draw a line from the definitions below to their terms.

23Giving up and losing only half of theoriginal bet. Doubling down.

24After being dealt two cards of thesame point value, to dividing theminto two separate hands.

Insurance.

25 Making an additional wager up to theoriginal amount. Surrender.

26 When one of dealer’s card is an ace,making an additional side bet. Split pairs.

7 Check all of the boxes that give you the best odds of winning in the given situation.

Page 25 of 333

Hit

Stay

Split

Double Down

Surrender (if allowed)

Dealer

Player

A

A

Page 26: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 26/33

27

28

29

30

Page 26 of 33

8

Hit

Stay

Split

Double Down

Surrender (if allowed)

Dealer

Player

10

8

10

Hit

Stay

SplitDouble Down

Surrender (if allowed)

Dealer

Player

8

9

8

Hit

Stay

Split

Double Down

Surrender (if allowed)

Dealer

Player

9

4

Page 27: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 27/33

8 Name each of the components of a Blackjack table by writing their names on the respective line (31 – 36).

Page 27 of 33

Page 28: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 28/33

R E A C T I O N N A I R EPlease, help us improve the quality of this workshop by filling out this reactionnaire.

Name: Date:

General Satisfaction Level:

Please rate each of the statements belowusing the following scale.

S t r o n g l y

d i s a g r e e

D i s a g r e e

N e u

t r a

l

A g r e e

S t r o n g l y a g r e e

1 The workshop was interesting.

2 The workshop was fun.

3 I understand the purpose of the workshop.

4 The content of the workshop is relevant tomy life.

Workshop Organization:

Please rate each of the statements belowusing the following scale.

S t r o n g

l y

d i s a g r e e

D i s a g r e e

N e u

t r a

l

A g r e e

S t r o n g l y

a g r e e

5 The session was held at a time during which Iwas able to focus my attention on the task.

6 I had adequate time to practice playingBlackjack during the workshop.

Please rate each of the following items by choosing the option that best fits your opinion.

7 Workshop length. Too short Adequate Too long

8 Visibility of the video. Difficult to see Adequate Highly visible

9 Volume level of the video. Too soft Adequate Too loud

10 The pace of the video. Too fast Adequate Too slow

Page 28 of 33

Page 29: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 29/33

Workshop Effectiveness:

Please rate each of the statements belowusing the following scale.

S t r o n g

l y

d i s a g r e e

D i s a g r e e

N e u

t r a

l

A g r e e

S t r o n g l y a g r e e

11 I can name the components of a Blackjacktable.

12 I can use terms related to the game of Blackjack.

13 I can identify all applicable rules given aBlackjack table.

14 I can use hand signals to indicate my desiredaction when playing Blackjack.

15After seeing my cards and the dealer’s up-card, I know what action should be takenbased on the strategies provided in the video.

Personal Perceptions:

Please rate each of the statements belowusing the following scale.

S t r o n g l y

d i s a g r e e

D i s a g r e e

N e u

t r a

l

A g r e e

S t r o n g

l y a g r e e

16 Blackjack is a game of luck.

17 Blackjack is a game of strategy.

18 I adhere to a particular strategy when playingBlackjack.

19 I would like to play Blackjack again.

20 I would like to visit a casino to playBlackjack.

Additional Comments:

Page 29 of 33

Page 30: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 30/33

Page 30 of 33

Page 31: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 31/33

Page 31 of 33

Page 32: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 32/33

Page 33: Final Report Blackjack 03-26-2004 This is the Goooood One

8/8/2019 Final Report Blackjack 03-26-2004 This is the Goooood One

http://slidepdf.com/reader/full/final-report-blackjack-03-26-2004-this-is-the-goooood-one 33/33