1

Instructional Design for Distance Learning 11/7/99By Al Byers

The primary source of the model discussed is one that is presented from chapter 3 of the bookDistance Training: How Innovative Organizations Are Using Technology to Maximize Learningand Meet Business Objectives by Deborah A. Schreiber and Zane L. Berge (1998). Thus, themodel presented is one that is tailored toward the business community in its strategic planning ofdistant and distributed learning environments. Other relevant journal articles and booksdiscussing additional distance variables such as learner characteristics, time, place and mediaattributes will also be incorporated into this model. Finally, with respect to the businesscommunity an evaluation component of instructional design for distributed learning will bediscussed from a learning effectiveness standpoint and Return on Investment principles (ROI).First at brief discussion of the logic behind applying an instructional systems design (ISD) todistance training will be presented.

The Logic:

The primary focus of any company training effort (distance or not) is to increase the desiredperformance outcomes as aligned with business goals and objectives taking into account theinternal corporate dynamics. By applying an ISD model to the design, development,implementation, utilization, management and evaluation of distance and distributed learning, oneprovides a method of comprehension for the interrelated roles between the student-culture,student-student, student-content, student-interface and student-instructor relationships at work.By utilizing an ISD model one is able to effectively identify all the interrelated contributingcomponents of a distance learning environment and determine the proportional affect of each,thus maximizing their collective impact (Moore & Kearsley, 1996). In other words, an ISDmodel allows the instructional design team to focus on (a) what the learner will be able to do as aresult of the distance delivered instruction, (b) the interaction between the students, content,technology and instructor, and (c) the replication of distance learning outcomes in support ofbusiness outcomes aligned with corporate goals (Dick & Carey, 1996; Moore & Kearsley, 1996).

Additionally, the ISD model will allow the instructional design team to analyze the intendedlearning outcomes as related to the type of skill, knowledge or attitude desired and leverage thestrengths of the delivery technology mode used. With the decreasing cost of more powerfulcomputers, and the increasing bandwidth of connectivity between distributed learningenvironments (DLE), the mode of distance delivery is becoming one of a hybrid approach,incorporating a mixture of technology solutions best suited toward their unique and individualattributes. For example, a DLE may consists of an asynchronous text based threaded discussionfor reflection and archiving of ideas, while incorporating the use of CD-ROMs for real-timedelivery of animation or digital movies. A live synchronous chat or desktop audio/videoconference may be used as a collaborative tool within a common shared virtual workspace toreach consensus on a group project or discussion of a case study/role-play, while theInternet/Intranet may be incorporated for content delivery, practice and feedback.

2

Finally, Schreiber and Berge (1998) emphasize the importance of a Distance Learning CoreSteering Committee (DLCSC) to ensure stakeholder buy-in at all levels of an organization andassimilation of the distance training program into the corporate culture. This DLCSC should becomprised of communication professionals, information systems engineers, executive managers,education performance experts and instructional designers. Now that the justification behindapplying ISD to distance training has been presented, a look at the Instructional Design Modelfor Distance Training (IDM-DT) developed by Schreiber and Berge will follow with a briefoverview.

The Instructional Design Model for Distance Training (IDM-DT):

Shcreiber and Berge (1998) describe their model as follows:

The Instructional Design Model for Distance Training (IDM-DT) correlates instructional strategy, deliverytechnology, and desired performance outcomes. The model is driven by an initial "gap analysis" (todetermine business needs), followed by an identification of associated performance objectives that make

1. AnalyzeBusiness Needs(Conduct GapAnalysis)

4. AccommodateTechnology Plan

2. IdentifyStrategicDistanceTraining Eventor Program

3. ApplyConceptualFrameworks ofLearning to DT

5. CorrelateInstructionalMat'ls/TechnologicalDelivery Tools

6. SecureImplementationSupport forBalanced Roll-out

7. EvaluateDistanceTraining andMeasureTransfer

Apply technologyselection matrix

Develop Instructional.Objectives to meet DTPerformanceOutcomes

3

strategic contributions to the organization. The IDM-DT provides a methodology for meeting the demandsof distance training instructional event(s) through applying conceptual frameworks of learning, usingselection criteria to identify effective technology delivery tools, and designing instructional materialsconsistent with the strength of the delivery tools. The impact of organizational culture and internalcorporate dynamics is also addressed (p.43).

It should be noted the model above is a reiterative one as displayed by the arrows connectingeach phase. There are 7 distinct phases and once one phase is complete it may be revisited oranalyzed in light of new information acquired from a later analysis. Now that the IDM-DT hasbeen presented a breakdown and elaboration of each step will follow.

Analyze Business Needs: Phase 1

Step one of the IDM-DT focuses on an analysis of business needs and goals of the organizationand how distance training can further these strategic objectives. To do this one should conductthe following tasks:

• Conduct an organizational analysis• Determine if a gap exists between what is desired within an organization and what

currently exist in the corporation• Confirm the primary business goals and needs against the desired organizational

change

The organizational analysis attempts to identify the traits and characteristics of an organizationthrough documented processes and procedures, as well as the "way things are really done on thefront line" within an organization. This is accomplished by live interviews and self-administeredsurveys from employees at various levels within an organization.

Next an "Organizational Gap Analysis" is conducted. The goal from the analysis is to identifywhat performances are occurring within a company and compare this to the specificperformances needed to meet overall business goals. The difference between company goals andthe actual observed goals determines the "gap" or "needs" of an organization. Again, liveinterviews and self-administered surveys can be used to document what are the current policiesand institutional procedures within a company. In addition to survey data, one may also look atquantitative indicators like "production levels, product reliability, fixed and variable cost, grossmargins of ROI, and response to customer service" (Schreiber & Berge, 1998, p.44).

Finally, before the "gaps" or business "needs" can be finalized, one needs to confirm theorganization's primary business goals and objectives. This can be achieved by looking at thefollowing areas (Robinson & Robinson, 1996):

• A company's mission statement or charter• The most recent annual report(s)• A company's 5-year long range planning document• An organization's operating statements• The information systems 3-year technology plan• Government and industry standards for the company's market

4

• Economic projections• Customer demographic reports• Key individuals responsible for securing business with desired markets

By confirming corporate goals an analysis of the "gap" can be scrutinized for appropriateorganizational performance changes that will help the institution reach its desired levels.Ultimately an alignment between the desired SKA of a company's personnel and theseorganizational "gaps" occur (Phase 3). Phase 2 will next be discussed.

Identify Strategic Distance TrainingEvents and Programs: Phase 2

Strategic contributions to an organization are those which make a company more competitive,more productive and more efficient. Distance training efforts should produce learningperformance outcomes in individuals that align and support a company's strategic businessneeds. To begin this analysis one needs to determine first if the corporate need is one of aperformance change in individuals that can be changed via training. If this is deemed the case,then one needs to ask which skill, knowledge or attitude requires improvement. Finally, once onehas identified the type of SKA to improve, then one needs to ask if this is something that can besuccessfully taught at a distance.

Thus phase 2 of IDM-DT has two components. First, one should "identify business-drivenperformance learning outcomes that may contribute strategically to an organization" and then"determine the potential return on investment (ROI) of distance training to attain the identifiedperformance outcomes" (Schreiber & Berge, 1998, p.45). Dick and Carey (1996) define aperformance outcome as that which the learner will be able to do as a result of interacting withthe instruction. It is the instructional event that executes the development, organization,sequencing, implementation, and evaluation of the ISM-DT to facilitate learning of the newskill(s), knowledge or attitude.

Determining Business -Driven Performance Outcomes:

To identify and develop business driven performance outcomes a process will be discussed thatidentifies the salient features of this phase of the design model. At this stage it is recommendedthat a corporate team assist in this ISD process. The team may be comprised of trainingprofessionals, organizational managers, content experts, and learners of the target population.

First the team should generate a list of all business needs as identified from the priororganizational gap analyses (phase 1), keeping in mind that the needs should be stated inmeasurable and observable terms (productivity levels, product reliability, etc.) Next a discussionidentifying the forces behind these needs should occur. Past training and non-training strategiesshould be incorporated into this analysis in attempt to solve these yet unanswered needs. Theteam should then try to incorporate the implications for performance improvement as identified

5

by the following individuals: clients, instructional designers, content specialists and performanceconsultants. Paralleling this discussion, the team should also identify all behaviors of staff andmanagers that would assist in improving the aforementioned desired performances as alignedwith business needs.

Once all the possible behaviors have been listed, the team should then select those behaviors thatmost efficiently and effectively contribute to meeting the strategic business needs identified inphase 1. Now tasks analyses can be executed by instructional designers and subject matterexperts that will break down or proceduralize the desired performances into their specific steps,subordinate steps, and entry level skills(Dick & Carey, 1996). Tasks analysis of organizational,procedural, contextual and cultural implications may also be conducted at this phase as they tooimpact the transfer of training into the workplace. At the conclusion of this effort the teamshould have clear statements of what the learner(s) will be able to do as a result of the instruction(performance outcomes) and how these outcomes contribute to the business needs of anorganization. The actual instructional analysis will be examined in detail of phase 3 of the IDM-DT. Suffice it to say the component of phase 2 should end with a list of performance outcomesaligned with business needs and goals. Next a brief description describing how to determine aCost-Benefit Analysis (CBR) and ROI projections for distance training will be discussed.

Determining Cost-Benefit Analysis and Return on Investment

This portion of the IDM-DT comes from the book: The Handbook of Training Evaluation andMeasurement Methods by Jack J. Phillips (1997). Chapters 13 and 15 were the source ofdiscussion below. It should be noted that the IDM-DT model of Schreiber and Berge (1998) isstill being adhered too at this point. The resource of Phillips (1997) is being used as a supplementto expand upon this phase of the IDM-DT model.

The concept behind Cost-Benefit Analysis and ROI does not focus on measuring theeffectiveness of your training from assessing individual learner outcomes, but measuring thebottom line financial value of a training efforts by comparing the overall costs of training to itsbenefits. If the result of training increases the financial health of a company, then training shouldbe viewed as an investment and not an expense. When training is viewed as increasing thefinancial wealth of a company, then the funding mechanisms for training are viewed asmandatory, similar to the expenditures for things like the purchasing or upgrading of equipmentand facilities. The cost/benefit ratio and the return on investment are the two most commonmeasures used to indicate a training program's financial worth to its company (Phillips, 1997).

Capturing cost of training (distance delivered included) is not a simple thing to do. But todevelop an ROI figure that is valid and reliable, costs accrued must be accurate and credible.Overstating or inflating an ROI figure for distance training will do more damage than good, andmay hinder future budget requests or lesson the integrity of the reporting department.

Since Cost-Benefit Analysis and ROI may be an arduous endeavor, it may be worth noting someof the additional reasons for determining distance training costs before delving into the details ofhow to determine cost-benefits and ROI.

6

Phillips (1997) cites several benefits for determining training program costs (distance included):• Calculating total expenditures for training (both face-2-face and at a distance)• Determine the relative cost of each training program• Predict future training costs• Calculate benefits versus cost for a specific training program• To improve the efficiency of the training program• To evaluate alternatives to a proposed training program• To plan and budget for next year's training budget• To develop a marginal-cost pricing system

So, in addition to determining ROI, assessing the costs of distance training serves manyadditional benefits. It should also be noted that when determining the cost/benefit ratio and ROIone needs to determine an annualized value or return time frame for the training in question. Thiscould be 1 year or more depending on the length of the training and the time for visible financialresults to be measured as a result of the training.

It is becoming accepted practice to use annualized values of 1 year for most training programs.Given that some observed financial impacts may not appear until year 2 or 3 after a trainingevent, using 1 year for an annualized return is a conservative approach (Phillips, 1997). Also ofutmost importance is the necessity in communicating how the cost/benefit ration and ROI aredetermined. Using both a conservative annualized value and communicating how one arrives attheir numbers lends credence to the results. A detailed look at the costs/benefits ratio and ROIwill now occur.

Cost/Benefit Ratio:

The Cost/Benefit Ratio (CBR) merely attempts to measure the annual economic benefitsof a training program against it overall cost. The formula for determining the CBR is asfollows:

Cost/Benefit Ratio = Program Benefits Program Costs

The CBR nomenclature usually is displayed as ratio, i.e. 3:1. This would mean that thebenefits were three times as great as the cost of the training program or that for everydollar invested in the training program 3 dollars in benefits were returned. For example, ifan annual training program cost $ 192,666 to administer but generated a benefit to thecompany of $578,000 the CBR would be:

CBR = $578,000 = 3:1$192,666

A CBR result of 1:1 would mean that the benefits of a program equal their cost. Thiswould not be a prudent investment.

7

The CBR analysis is a special financial benefit measure that allows its worth to remainunique as compared with standard benefit measures for things like equipment orfacilities. Some executives prefer to use different methods to compare return on traininginvestments as compared with other corporate investments. In this regard there is not anoverall accepted standard value for CBR results. In some organizations a minimum of1.25:1 ratio is required (Phillips, 1997).

Phillips goes on to specify that training cost should be broken down into the followingcategories: analysis cost, development costs, delivery costs and evaluation costs. Bydoing this one is not only able to determine CBR, but also the relative and future cost ofprojected training programs as discussed at the beginning of this section (1997).

The following is a list of cost expenses one should include when attempting to measureall cost associated with a training effort (Phillips, 1997).

• Salaries and Benefits (training personnel)• Salaries and Benefits (Other company personnel assisting in training effort)• Salary and Benefits (Participants while taking training)• Meals, Travel and Incidental Expenses (training personnel)• Meal, Travel, and Accommodations (participants)• Office Supplies and Expenses (stationary, services, subscriptions, postage,

telephone, etc.)• Program Materials and Supplies (films, binders, hand-out, purchased

programs)• Printing and Reproduction (of training materials)• Outside Services (fees and expenses to those performing special consulting or

production services as related to training efforts)• Equipment Expense Allocation (amortized over life of equipment for owned

equipment like computers, VCRs, etc.)• Equipment (rented)• Equipment (maintenance of owned equipment)• Registration Fees (conference and registration fees and fees for trade

membership association for training employees)• Facilities Expense Allocation (should cost be incurred for use of company

owned facilities)• Facilities (Rental expenses for facilities)• General Overhead Allocation (prorated to training programs)• Other Miscellaneous Expenses (those not provided for elsewhere)

Phillips (1997) then provides comprehensive worksheets and tables that break outspecific charges for training as related to each of the four areas previously mentioned:analysis costs, development costs, delivery costs, and evaluation costs. It is beyond thescope of this model to discuss the expense account allocation procedure Phillips provides,

8

suffice it to say that an accurate accounting of all the cost incurred from training iscompletely possible.

With respect to forecasting the financial benefits of a distance training program, oneneeds to make predictions concerning the increased efficiency, accuracy, or quality ofimprovement that result from the trainig effort. As stated earlier, past records of trainingresults can aid in this area. Ultimately, the value of the training will be born out throughactual productivity returns upon the conclusion of the training (discussed in phase 7 ofthe IDM-DT). At this phase of the design process though it is essential to projectconservative financial benefits as a result of the training, for this will help determinr theworth of implementation of the distant training program.

Once the CBR is determined, one then has the figures to determine the ROI for trainingefforts. It should be noted that the discussion of late has minimally referred to distancetraining efforts. This is intentional, as the financial measures discussed pertain to alltraining efforts. A discussion about how distance training specifically fares with respectto CBR and ROI measures will follow the ROI explanation.

Return on Investment (ROI):

The Return on Investment is nothing more than a percentage of return determined bydividing the net program cost by the overall program cost and multiplying that fraction by100.

ROI = Net Program Benefits X 100Program Costs

The net program benefits are merely the program benefits minus the program cost. Youshould already have these figures from determining the CBR above.

For example, taking the same figures used previously (program benefits of $578,0000and program cost of $192,666) the ROI would be:

ROI = ($578,000 - $192,666) X 100 = 200 % $192,666

Thus, for every dollar spent on training a return on investment of $2.00 would occur.Comparing the ROI to the CBR determined earlier, one could see they are related by afactor of 1, with the ROI being a more conservative measure. The real benefit ofdetermining ROI is that one can now compare the value of training on an equal measureagainst other company investments. In other words, while CBR is a unique financialmeasure exclusively used for training, ROI is determined in the same manner as otherinvestment returns for a company.

9

Phillips (1997) also provides ROI processes that aid one in determining ROI from variouslevel's of training evaluation as specified by Donald Kilpatric's book: Evaluating TrainingPrograms: The Four Levels (1998). The four levels are: (1) Reaction, (2) Learning, (3)Behavior (3), and (4) Results (Kilpatrick, 1998, p. 19). Phillips discusses how ROI can beestimated from asking questions on exit surveys and action plans of the lowest type ofevaluation-level 1 (concluding at the end of a training program). While level 1 is thelowest or least valid measure of evaluation, should other more rigorous numerical-basedlevels of evaluation be unobtainable, level 1 ROI is better than no analysis whatsoever.Should a further explanation of determining ROI be desired, reference Phillips' (1997)and/or Kilpatrick's (1998) book(s) on the subject. To finish Phase 2 of the IDM-DT, abrief explanation of ROI will be addressed specifically from the standpoint of distancedelivered training.

Distance Training ROI:

Whalen and Wright (1999) in their article "Methodology for Cost-Benefit Analysis ofWeb-Based Tele-Learning: Case Study of the Bell Online Institute" describe key designelements that should be considered when evaluating the cost effectiveness of online web-based training. In their ROI comparisons, one is not comparing the CBR value toward thebottom line of the company's earning revenue (as discussed above), but analyzing thebenefits of web-based delivery versus other methods of delivery. They compared the costbenefits of distance delivered training (both synchronous and asynchronous) to face-to-face instruction.

The costing methodology that follows was developed in a pilot study by Bell Canada inefforts to analyze the financial benefits of web-based training as an alternative toclassroom instruction. The first measure is called the "Breakeven Point" and is defined asthe point at which development and delivery cost are recovered. This Breakeven Point isfurther defined as the number of students that must be trained via web-delivery such thatthe cost of delivery per student is less than the per student cost of delivery for classroomtraining. The premise is that web-delivered courses have a higher initial development costcompared to classroom instruction, but once deployed, the delivery cost per student issignificantly less after the "Breakeven Point" is reached. See figure 1 below for a visualrepresentation of the "Breakeven" concept (Whalen & Wright, 1999).

10

Whalen and Wright determined the ROI for an identical course delivered using thefollowing synchronous and asynchronous web-delivered training tools: WebCT, Mentys,Pebblesoft, and Centra Symposium. A brief look at how these cost were compared willnow follow.

To compare these costs, Whalen and Wrght (1999) analyze expenditures from both fixedand variable costs as well as by cost for each type of course. Fixed cost are things like thepurchase of hardware and the licensing of software, while variable cost are thosedependent on the frequency of distance training events, the number of students receivingtraining, and consumables expended during each training session (travel,accommodations, meals, materials, salary and benefits of participants and trainers, etc.).For a more detailed list of training cost, refer to "Cost/Benefit Ratio" section referencedearlier in this paper.

The study concludes by finding the cost of both asynchronous and synchronous web-based instruction initially more expensive to develop as compared to face to faceinstruction. Although, in the long term (given a course life of 3-5 years), web-based costoverall were less expensive than classroom instruction given the decrease in variable costof repeated course delivery.

In summation, when one is determining the CBR and ROI for distance training, inaddition to determining the overall value toward the company's profit statement, oneshould also analysze the financial impact as discussed above with respect to deliverymode (web-based, interactive video conference, face to face, etc.). With phase 2 of theIDM-DT discussed and strategic distance training events and programs analyzed it is timeto move on to phase 3 of the IDM-DT model.

Total FixedCost

Classroom

Web-Based

# of Students

Breakeven Numberof Students

Figure 1. Breakeven Number of Students for web-based training ac compared to classroom instruction(Whalen & Wright, 1999)

11

Apply Conceptual Frameworks ofLearning to Distance Training: Phase 3

In order for distance training to be successful one should next perform an instructional analysiswhich selects an appropriate instructional strategy as supported by a specific model of learning.The goal of an instructional analysis is to align an instructional strategy that facilitates themastery of company driven performance outcomes as identified in phase two of the IDM-DT.

Implementation of Instructional Analysisfor Design of Strategic Distance Training Instructional Events

The instructional analysis process is multifaceted and has the design team consider the followingareas: sequencing of instructional concepts, size of the instructional module (chunking), style ormode of presentation and types of examples (both good and bad) used to illustrate the conceptsin question.

Schreiber and Berge (1998) provide the following sequencing of events to begin the instructionalanalysis:

1. List the business driven performance outcome(s) determined in phase 2.2. Classify the type of learning performance outcomes according to following learning

taxonomy (Bloom, 1984):• Verbal information (declarative knowledge)• Intellectual skills (procedural, conceptual, rule application or higher order

problem solving)• Psychomotor skills (execute, perform, etc.)• Attitudes (intrinsically motivated choice made by learner)

For a more detailed listing of Bloom's taxonomy refer to appendix 1.

3. Analyze the target audience in the following areas: dominate learning style(s),prerequisite skill levels, motivation, and experience with delivery technologies. Here tooone should describe the size of the audience, the remote site locations, and access to thetype of technologies at each remote site and any assistance that may be available (sitecoordinator, help desk, email support, FAQ pages, live chats, technical, institutional andprogrammatic support, etc.).

4. Revisit the earlier task analysis performed in phase 2 and revise if/as needed. One shouldremember to also reexamine the contextual, procedural, and organizational cultureanalyses executed in phase 2 for the implications to distance training program. As statedearlier in the paper, a Distance Learning Core Steering Committee (DLCSC) will beessential in garnering buy-in from stakeholders at all levels of the organization andprovide keen insight into the business context and culture in which the training beinstituted.

5. Apply Gagne's taxonomy of learning conditions to identify instructional strategies for thepreviously identified performance outcomes (1998). Gagne' provides a comprehensive

12

listing of specific strategies and conditions to facilitate the learning of knowledge,intellectual, attitudinal, and motor skills. The application of research-based learningstrategies is essential to successful instruction of any type.

6. Analyze the strengths and weaknesses of various instructional methodologies (lecture,role-play, case study, collaborative group projects, student presentation, discovery labs,debate, etc.) and select the methods that will provide the essential levels of interactionand dialog needed between student-student and student-instructor whether synchronousor asynchronous, depending on the type of learning and appropriate congnitive domain(Weston & Cranton, 1986). For a detailed listing of cognitive domains as identified bylearning method, refer to Weston & Cranton's article. Also included in this analysis stepis the examination of degree to which the learner will be given control over the pace anddepth of instruction.

Characteristics of Instructional Events for Distance Training

This section of the instructional analysis discusses the individual components of a distance-training event, which are the introduction, the activities, practice and feedback, review,assessment and transfer. A brief description of each section will next be described.

1. Introduction or orienting functions of an instructional event may include one or moreof the following (Merrill & Tennison, 1994):

• Gain learner attention• Articulate in some way the SKA1 already needed to succeed within the new

learning environment• Identify opportunities in which learners will relate what is about to be learned

(goal) to what they already know how to do• Inform learners of objectives (o.k. to be vague)• Fit objective(s) into a "Big Picture"

• Present the utility (relevance) of the SKA to be learned

• Clearly identify the incentives/rewards for learning the SKA and succeedingwithin the learning environment

• Establish clearly-perceived learner accountability, role(s) and task(s) within thelearning environment

• Establish clearly-perceived instructor role(s) and learner support mechanisms

• Employ specific content-area methodologies if applicable (ensuring that all otherelements are properly addressed)

The main purpose of the introduction is to capture the learner's attention, buildinginitial confidence in the ability for the learner to succeed in the impending trainingevent, and to demonstrate the relevance and value of the training to learner.

13

2. The Activity or instructional component of the training may involve one or more of thefollowing (Merrill & Tennison, 1994):

• Establish in context the appropriate conditions for the type(s) of new SKAfacilitated

• Relate all SKA to a clearly-defined content domain (subject or area which trainingis serving)

• Provide learner guidance for learners as they apply information presented incontext to SKA being facilitated

• Present a variety of clear, concrete examples and nonexamples in context• Provide opportunities for learners to explore the learning environment with

minimal instructor guidance and intervention.

The activity component of the instructional event is where the participants initiallyinteract with the content information and possibly other learners. In a distancetraining situation, interaction is paramount to overcome transactional distance and thepsychological and social feelings of isolation (Moore, 1996; Wagner, 1997;Wolcott, 1996). Interaction in fundamental to the success of distant educationprograms as stated earlier and should be intentially structured into distance learningenvironments. (McIsaac & Blocher, 1998).

Thus, with repsect to interaction the design team needs to ensure that opportunitiesare provided for distance learners to contribute, respond, share and reflect upon thecontent being presented. Interaction during this component of distance training mayincorporate questioning strategies, group presentations, case studies, role-playing,storytelling, simulations, etc. (Weston & Cranton, 1986) "The Interactivity GuidePyramid" may be a tool used at this phase of the design process. It presents variedactivities that facilitate a balanced mix of instructional interaction and may be used atthis phase (Mantyla & Gividen, 1997).

Finally, should instructors be presenting in a synchronous environment, they shouldgo no longer than 5-7 minutes before including learner participation, and should varythe media type presented throughout the synchronous session (PowerPoint, video,overhead illustrations or flow charts, f2f discussion, etc.).

3. The Practice/Feedback Instructional component should involve the following (Merrill& Tennison, 1994):

• Provide initial learner guidance• Must match performances and conditions indicated within objectives and

presented in context

• All should get practice• Feedback as immediate as possible (unless delayed feedback is desirable)

14

In a distance training environment, the design of practice and feedback allowsstudent-instructor interaction to occur and engages the learner in the rehearsal of thecontent. Mere presentation of content is only the beginning of facilitating learning. Inorder for the content to be internally processed and assimilated from inert to appliedknowledge, the learner must utilize the new SKA presented, elaborating and"practicing" the intended outcome, receiving reinforcement for erroneous answers(Wagner, 1997).

In a distance training environment this should be facilitated on an individual andgroup basis. Should collaborative groups be used the instructor needs to structure thegroups such that positive interdependence and individual accountability occurbetween and among group members (Brush, 1998). With the wealth of distancecommunication tools available today (asynchronous threaded-discussions, live chatsessions, live 2-way audio, virtual whiteboards, and/or point to point desktop videoconferencing), the design team has a wealth of tools to foster interaction andcollaboration.

4. The "Review" Instructional component would involve these elements (Merrill &Tennison, 1994):

• Provide opportunities for learners to summarize the key ideas (including howthese ideas fit into the "Big Picture”), what they learned how to do, and how theypersonally learned it

• Restate objectives

This section of the distance training event again allows interaction on the part of thelearner and facilitates a meta-cognitive or internalization of the intended learning.Student reflection in a distance environment is easily facilitated via text, audio orvideo communication tools.

5. The Assessment component of a training event should match performances and conditions indicated within objectives with the terminal goal possibly being the only thing assessed (Merrill & Tennison, 1994).

If clear concrete and measurable instructional objectives were initially developed,then the assessment component of a distance training event has already beengenerated. For the assessment should match the instructional objective.

Although in a distance environment, depending on the type of desired performance,structuring the assessment in context for verification may take some advancesplanning. For example, if the instructional objective were for the learner to import ascanned image into Photoshop and save the output file as a JPEG image, one wouldneed to determine a method to evaluate attainment of this objective at a distance. In adistance training situation the assessment could occur in any of the following ways:

15

• Via a proctor monitoring the student completing the procedure• Asynchronously video tape the student performing the task and mailing the

tape to the instructor• Using desktop videoconferencing and application sharing, allowing the

instructor to monitor the student performance in real time

6.The "Transfer" component of a distance training event should (Merrill & Tennison, 1994):

• Present new context which elicits the same performances under differentconditions

• Make the utility of succeeding within the new context apparent• Present cues within the new context which aid learners in selecting and

applying the appropriate previously-learned SKA• Clearly identify the incentives/rewards succeeding within the new context

In distance training, transfer is paramount. If the new SKA learned are acquired in adistant distributed classroom setting, but intended to transfer to the factory floor oroffice cubicle, then facilitation of transfer is essential. One way to facilitate transferfrom the distant training environment to the workplace is to design the activity andpractice components of the training event to approximate or incorporate the "actual"environment in which the desired SKA will be applied (referenced as "in context" incomponent 2 and 3 above). In essence the "context" in which the SKA are intendedfor transfer are simulated during the distance learning event (Tessmer & Richey,1997).

Finally, throughout the instructional event, the design team should incorporateKeller's ARCS model of motivation for strategies to hold the learners interestthroughout the instruction (1987). Keller defined four areas of motivation: Attention,Relevance, Confidence and Satisfaction. His research found that while theintroduction of instruction may initially gain the learner's attention, unless the learneris demonstrated the other stages of motivation (relevance, confidence andsatisfaction) as he/she is progressing through the event, the learner may loose interestand fail to complete the instructional objective desired (Keller, 1987). See Appendix2 for a more detailed overview of Keller's model.

In summary, distance training that identifies and reinforces a particular model of learningwill ultimately be more successful than those that do not. Interaction is paramount acrossall components of distant training and should be intentionally designed into theinstructional event (Wagner, 1997) .

Before leaving this phase of the IDM-DT a brief look at the interaction between thestudent-interface will be presented. Previously above the value of student-student,student-instructor and student-content interaction was discussed. Student-interfaceinteraction is that which takes place between the student and the technology used todeliver the distance instruction. Technology at its best would be completely transparent tothe learner. While this is usually not the case, the interface should not intentionally

16

impede or detract from the learning event, but facilitate the transfer of knowledge andprovide minimal interference (McIsaac & Blocher, 1998). Properly designed technologymay even facilitate learning that would otherwise be difficult or too dangerous to teach(simulations and virtual 3-D representations). With phase 3 not presented the discussionof the organizational technology plan will follow.

Accommodate (Organizational)Technology Plan: Phase 4

As stated earlier there is a wealth of distance collaboration tools available today. It is usually notthe lack of technology that impedes successful distant training, but the lack of foresight inplanning for the procurement, utilization, and maintenance of technology via a distancetechnology plan. Phase 4 will provide suggestions for the development of an effectivetechnology plan to support distant training programs.

Schreiber and Berge (1998) suggest that there are two approaches that affect the purchase andutilization of technology. One approach is driven by the instructional design needs of theorganization, while the other approach is driven by the organizational and business needs of acompany. Thus designers may request technology based of its ability for real time high fidelitycollaboration, while the executive purchase officers of a company may scrutinize the purchase oftechnology for its return on investment.

Business-Driven Procurement and Utilization of Organizational Technology forDistance Training

Often instructional software enterprise solutions and instructional designers areconsidered marginal cost for a company, who instead look to the instructional systems orthe computing services department to fulfill this need. Also utilization of technology forinstructional purposes may compete against other limited company resources (Green,1997; Schreiber, 1996). This should not be the case and training should be considered aninvestment for the company, returning or contributing profits toward the bottom line.

Schreiber and Berge (1998) reviewed the research and found that two solutions have beensuggested to combat this procurement and utilization problem, the are:

1. Develop and implement an Organizational Technology Plan2. Create and recognize and interdisciplinary Distance Learning Core Steering

Committee

Organizational Technology Plan:An organization's technology plan basically provide a guiding document thatdetermines the procurement and utilization of technology providing policies andprocedures for allocating resources, determining cost benefits, and controllingbudgets.

17

Schreiber and Berge (1998) researched the literature and found that a technology planshould contain the following:

1. A description of how technology will support corporate goals for the next3-5 years

2. Alternative technologies statement and how each will support corporategoals

3. Distance training statement recognizing the department as a unique entityand how/who it supports the within the company

4. Guiding principles statement delineating the improvement of technologyto support the changing and growing needs of the business

5. An alignment of strategic company goals and how they are directlysupported in part by technology utilization

6. Identify core business costs (intrinsic to doing daily business), andmarginal costs (those not documented), and bundled cost (those bundledinto core business cost)

7. Line item budget allocation for the following distance trainingexpenditures

• distance training course design• materials development• distance delivery skills training• computing services' support of distance learning participants• computing services' support of integration of distance instruction

8. Financial plan for amortization and replacement of computers, softwareand hardware

9. Defined role for information systems support in distance learning efforts

Distance Learning Core Steering Committee (DLCSC):The primary goal of the DLCSC is to support the utilization of organizationaltechnology, maximizing it effectiveness and facilitating it ongoing development.As the needs and goals of a company evolve, the DLCSC helps to ensure thecontinued interdisciplinary continuity and augmentation of technologies tosupport the strategic missions of the company.

The DLCSC is usually made up of representation from the following corporateemployees: executive management, information systems and/or network services,broadcasting or communications, instructional designers, and training andperformance consultants. Additionally temporary members may be included forthe implementation of new technology, which may be subject matter experts andend-users.

18

Instructional Design-Driven Identificationand Selection of Technology for Distance Learning

Schrebier and Berge (1998) state that as the DLCSC facilitate the acquisition andimplementation of supporting technology for distance training, making decisions aboutwhat attributes of various technologies may best support the company's identified trainingevents and delivery of instruction. This sub-phase of the IDM-DT is often debated amonginstructional designers about when it should occur in the design process. Some feel itshould be considered first, prior to the development of instructional events, while otherfeel the media doesn't matter, and the specific technology employed can be determined ata later time. Also, as stated earlier, a single mode of technology delivery may not be themost advantageous approach to distance training, but a hybrid approach, incorporatingvarious media, based of their unique attributes.

For this discussion, the author is assuming this document is intended to assist a mid-to-large size corporation that is concerned with implementing or reevaluating their currentdistance training efforts. Thus, it will be assumed that a variety of technologies (whetherowned or rented) may be available for distance training efforts. In this regard, one shouldnot restrict the instructional design process (phase 3) by selecting a single technologymedium, but decide which technologies are most appropriate given the attributes andavailability of a variety delivery mechanisms. The type of technology used can enhancethe type of learning and instructional method/strategy desired .

Schreiber and Berge (1998) state: "Maximizing utilization of technology for distancetraining is characterized by a clear concise understanding of instructional goals andperformance outcomes and subsequently designed instructional events and instructionalmaterials consistent with the chosen medium. The key is to work with rather than againstthe capabilities of the technology (p. 56). Given that each technology has uniquestrengths and weaknesses, an assessment of various technology usability characteristicsshould be examined. Schreiber and Berge (1998) provide a "Technology SelectionCriteria Matrix" that compares the following delivery mechanisms: Internet/Intranet,satellite, compresses video, CD-ROMs/DVDs, audiotape/videotape and printed materialagainst the following learning characteristics: access, audience, cost (communication),cost (start-up), fidelity (audio/video), instructional adaptability, instructionalmethodology, instructional strategy, interaction level, learning outcome, scheduling(synchronous/asynchronous), support service needed, and update capability. SeeAppendix 3 for a more detailed look at this matrix. With phase 3 now discussed, phase 4of the IDM-DT will next be presented.

Correlate Distance Learning Instructional Materialsto Technology Delivery Tools: Phase 5

Similar to the phase 4 sub-component of aligning technology to the instructional design events, acorrelation of instructional materials is similarly related. Phase 5 of the IDM-DT describes the

19

effective design and integration of materials to the technology used for delivery as related to twotraining modes: interactive videoconferencing (TV) technology and internet/intranet technology.

Interactive Videoconferencing Technology (IVC):Permitting live two-way audio and video communication, IVC technologies facilitate thelearning of both visual and auditory learners if the instructional materials used aredesigned properly. Visual materials may constitute slides, graphics, charts, videotapes,and 3-D VRML objects and should display the essential and salient elements of thecontent in question (Alessi & Trollip, 1991). Detailed in depth information may bedistributed via a printed handout.

Also text transmitted on screen should be of at least 36 point size and of sans seriftypeface. The use of bold, capitalization, and colored fonts may be used for emphasis, butavoid using all 3 simultaneously. Additionally, limit the use of multiple fonts within aslide presentation.

When document cameras are employed the layout of diagrams should align with the 4:3ratio employed by the television layout (horizontal rectangle). Additionally, the lensfocus should maximize the detail in the diagram and the zoom ability of the camerashould appropriately frame the intended field of view transmitted by the camera.

The audio attributes of IVC should be of high fidelity and ample volume. Microphonesshould be positioned appropriately and input levels adjusted accordingly. Shouldvideotape information be transmitted via IVC, the input level of the local microphonesshould be adjusted downward to avoid feedback and echo effect. Next the salient featuresof Internet/intranets will be examined.

Finally, when generating materials for presentation using IVC, the design team shouldconsider the complexity of the construct being taught. Weston & Cranton (1986) suggestapplying "Dale's Cone of Experience" and Torkelson's (1974) "Concept Cone" to helpdetermine the type and complexity of the transmitted images. Basically, if the learner hasno prior experience with the concept being taught then concrete and realistic imagesshould be used (graphics, diagrams, etc.). Conversley, the more experience a learner haswith the intended concept or the more abstract the concept being taught is, then moreabstract the visual symbols may be that are transmitted via IVC (text, symbols, etc.)(Weston & Cranton, 1986).

Internet/Intranet Technology:There are a variety of electronic instructional materials that are suited for web-based orCD-ROM delivery (text, graphics, streaming audio/video) and can provide a high level ofstudent-content interaction if structured properly. Student-student and student-instructorinteraction is also possible both asynchronously and in real-time.

20

The asynchronous attributes of web-based or CD-ROM delivery allows a high degree oflearner control as related to pace and depth of content information. For example, postingof content-based lectures may be done in text-based or graphic slide format, allowingquestions to be submitted to the instructor to answer questions as they arise. Listservsallow the dissemination of materials and immediate notification of announcements.Increases in bandwidth connectivity and the power of computers are also facilitating real-time desktop conferencing capabilities over TCP/IP networks, thus helping to decreasethe learner isolation sometimes inherent in self-paced learning environments.

Several potential limiting factors of internet/intranet based learning environments need tobe considered. First computer hardware and software platform compatibility issues mayimpose delivery problems, and should be addressed in the organizational technologyplans (phase 4). Similarly, dependent on the amount of multimedia incorporated intointernet/intranet based training, computers need to have appropriate RAM or audio/videoboards installed and necessary bandwidth connectivity.

Second, as referenced at the conclusion of phase 3, instructional designers need to beaware of the student-interface interaction and its level of potential hindrance/help tolearning. In intranet/internet learning environments then, the instructional design teamneeds to consider the need for "just in time" or dedicated interface training should it bedetermined necessary.

With phase 5 now presented, a look at the support for implementing distance-basedtraining will be addressed.

Secure Implementation SupportFor Balanced Roll-Out Strategy: Phase 6

Organizations that deploy successful distance training events secure the coordinated effort ofmultiple levels of the organization. This is accomplished or facilitated primarily by the efforts ofthe Distance Learning Core Steering Committee (DLCSC) and its multifaceted hierarchicalmembership referenced earlier in phase 4. Phase 6 of the IDM-DT will discuss the processesnecessary for proper implementation of a distance training program at the institutional level.

Schreiber and Berge (1998) discuss the mandatory execution of certain sophisticatedorganizational behaviors as identified in the organizational technology plan:

1. Proactive institutional planning for design, development and delivery of distancetraining

2. Organizational recognition of distance training as a unique and distinct component ofa company's training efforts with its own line item budget requirements

3. Availability of organizational computing services support for implementation andtechnical support, and evaluation

21

4. Collaborative contributions by interdisciplinary teams made up of subject matterexperts and end users

Leading the implementation efforts, the DLCSC must be proactive in predicting which facets ofthe organization may be most effected by the distance training roll-out, securing theircommitment and understanding of the value of the impending plan. Potential disruptions inworkflow my initially appear which need to be alleviated expeditiously and addressed from bothan organizational and cultural impact (analyses determined in phase 2 of the IDM-DT). Thisdisruption may also be alleviated by the flexibility of information systems and computer servicesgroup during the implementation phase.

Next, the DLCSC must monitor the implementation process, observing potential disruptions notonly in technical difficulties (as referenced earlier), but also with respect to unanticipatedchanges. For example, budgetary allocations or organizational support of the distance trainingeffort may be augmented mid-stream by executive management. Finally, the DLCSC must makea concerted effort and strong application of "Facilitation Processes" that:

1. Increases the ease of transition2. Lowers the intensity of opposition or obstacles3. Provides ongoing stimulation of both 1 and 2 above

Additional tasks during this time may included the adjustment of the "pace" of theimplementation roll-out and the creation of incentives for short and long term milestones for thesupporting organizational departments assisting in the implementation (computer services,network services, etc.). The DLCSC may also review the organizational technology plan, editingand reinforcing components as necessary with the interdepartmental committee liaisons. Inconclusion, the goal for this phase of the IDM-DT is to establish a stable but flexibleimplementation processes that is reflective throughout the implementation phase. The final phaseof the IDM-DT will now be presented.

Evaluate Distance Training Processesand Measure Transfer: Phase 7

The final phase in the IDM-DT defines how to evaluate distance training events as well as theinstructional design model for distance training. Measuring transfer of performance objectives insupport of strategic business goals is also addressed. To successfully evaluate a distance trainingprogram, including the individual training events, one must conduct formative as well assummative evaluation. The formative evaluation analysis below has been provided as an additionto Schreiber and Berge's IDM-DT model and comes from the book titled: Planing andConducting Formative Evaluations (Tessmer, 1998).

22

Formative evaluationFormative evaluation deals with the assessment of instructional materials andcomponents during their development stages and their effectiveness as used with varioustechnology delivery modes. Thus will presented during phase 7 of the IDM-DT,formative evaluation occurs during any phase concerning the development ofinstructional materials or events.

The primary intent of formative evaluation is to collect data for the purpose of improvingthe effectiveness of the instruction via revision. There are 4 primary types of formativeevaluation, each dealing with different areas of the instructional design process: expertreview, one-on-one learner review, small group evaluation, and field or pilot testing.

1. Expert ReviewsConducted by various experts to evaluate the instruction from different pointsof view. The following is a list of the different types of expert review possible.

• Instructional design expert-integrity of instructional designcomponents and objectives

• Content or subject-matter expert-review for clarity, quality andcompleteness

• Production (technology) expert-analyze the compatibility,development and deployment of technology

• Subject sophisticate (average student who has successfully completedsimilar instruction)-review for procedural completeness of subordinateskills and goal steps of instructional objectives as well as generalmotivational ability

• Teaching/training expert-review materials for analysis ofimplementation, feasibility and motivation

2. One-on-one learner reviewConducted by learners of various skill and ability levels from the targetpopulation. The instructional designer sits down one-on-one with learnersfrom the target population and reviews the instruction by interviewing thelearner for:

• Content clarity• Clarity of directions• Completeness of instruction• Difficulty level• Quality• Typographical/grammatical errors• General motivational appeal

3. Small Group Evaluation (SGE)The small group evaluation attempts to verify how the instruction would farein an authentic setting. Conducted by the instructional designer, the SGEattempts to confirm positive aspects of the instruction as well as seeking

23

improvement for implementation and administration. The small groupevaluation is usually administered to anywhere from 5-20 participants anddone so with the help of the future instructional trainer. The small groupevaluation is intended to collect data in the following areas:

• Time• Instructional delivery and implementation concerns (instructor’s role,

media/technology concerns etc.)• Performance on individual practice items• Performance on individual posttest items (congruency)• Attitudes from survey and interviews• Social/collaborative environment concerns

4. Field or Pilot TestingThe field trial represents the first evaluation of the instruction in a real setting.All material is evaluated with revisions made from the previous small groupevaluation and implementation procedures are scrutinized for efficiency andeffectiveness.

The data from a field trial stage are similar to that collected during thesummative evaluation (performance outcomes and attitudes). Field trialsanswer the following questions:

• Does the distance training solve the intended performancedeficiency

• Does the distance training program solve the strategic businessneeds identified in the organizational analysis in phase 1

Summative EvaluationSummative evaluation is the final evaluation of a distance training program and attemptsto determine the overall effectiveness and efficiency of the distant instruction. Did theprogram meet its intended goals? Does the learner now possess the intended SKA desiredas a result of interacting with the instruction? Are the SKA learned via distance trainingbeing transferred and implemented in the workplace? Summative evaluation involves thesystematic collection of data over time via qualitative and quantitative observations fromvarious levels within the company. Summative evaluation also determines the actualvalues for the cost/benefit ratio and return on investment projections generated in phase 2of the IDM-DT. Thus, when new distant training programs are suggested, past resultsfrom prior summative evalutated programs may be applied as a measure of potentialsuccess.

24

In Summary:

The IDM-DT is the systematic theory and practice of the design, development, implementation,utilization, management and evaluation of the processes and resources for distance instruction.There are seven distinct and reiterative phases that comprise the IDM-DT, each essential for thesuccessful creation a distance learning program. By initiating a Distance Learning Core SteeringCommittee to create an organizational technology plan and facilitate the implementation of thedistance training program as aligned with the strategic needs and goals of the organization, theprogram is better poised to achieve its intended goals.

25

Appendix 1

Classifying Performances

Category Description of Performance Outcome Common Verbs

VerbalInformation(DeclarativeKnowledge)

Verbal information the students must state,including: facts, dates, people, names, principles,generalizations etc.

State, Recite,Tell, Declare, Name,List, Define

[Bloom:Knowledge]*

Intellectual Skills:(ProceduralKnowledge)

DiscriminationsDistinguishing objects, features, or symbols (smoothversus not smooth, for example)

Distinguish,Differentiate,

Intellectual Skills:Concepts

Concrete Concepts:

Objects (parts of the body, for example),Classes of objects (plants, cells, etc.), Object features(5 arms, red, etc.), andObject relations (above, near, etc.)...that can be pointed out and identified.

Defined Concepts:

Objects, principles, classes, features, and relationsthat cannot be identified by pointing them out. Theymust be defined.

Examples include:"life", "energy", "evolution"

Identify,Label

Classify instances,Sort,Categorize

[Bloom:Comprehension]

Intellectual Skills:Rules

Rules make it possible to do something using symbols (mostcommonly, the symbols of language and math).

Rules include the application of single principles to explain,describe, or predict phenomena or events.

Rules make it possible for students to respond to a class ofthings with a class of performances.

Solve, Show,Demonstrate,Generate, Develop,Create, Determine,Calculate, Predict

[Bloom: Application]

26

Intellectual Skills:Higher-Order Rules(Problem Solving)

Higher order rules employ more than one rule or principle tosolve problems, perform tasks, or explain, describe, andpredict phenomena or events.

Students must decide which rules or principles must beutilized to perform tasks or explain, describe, or predictphenomena or events.

Solve, Show,Demonstrate,Generate, Develop,Create, Determine,Calculate, Predict,Defend, Support

[Bloom: Analysis,Synthesis,Evaluation]

Motor SkillsMotor skills represent physical activities requiringmovement and coordination of all or part of the body.

Execute, Perform,Swim, Walk, Run,Climb, Drill, Saw

AttitudesAttitudes represent intrinsically motivated choices peoplemake. Some of the most important outcomes are reallyattitudes.

Choose, Decide,Participate

[Bloom: Evaluation ]

Bloom, B., et al (1984). Taxonomy of educational objectives, Book 1: Cognitive Domain.White Plains, New York: Longman.

Appendix 1 was generated as an electronic instructional handout for a graduate class inEducational Evaluation offered by the Teaching and Learning Department of the College ofHuman Resources and Education at Virginia Tech located in Blacksburg, Virgina by Dr.Gregory Sherman, 1998.

27

Appendix 2

Keller’s ARCS Model of Addressing Motivation in the ID Process

Attention StrategiesIntroduce a fact that seems to contradict the learner’s past experience.Present an example that does not seem to exemplify a given concept.Introduce two equally plausible facts or principles, only one of which can be true.

Incongruity,Conflict

Play devil’s advocate.Show visual representations of any important object or set of ideas or relationships.Give examples of every instructionally important concept or principle.

Concreteness

Use content-related anecdotes, case studies, biographies, etc.In stand up delivery, vary the tone of your voice, and use body movement, pauses, andprops.Vary the format of instruction according to the attention span of the audience.Vary the medium of instruction.Break up print materials or (displays) by use of white space, visuals, tables, differenttypefaces, etc.Change the style of presentation.

Variability

Shift between student-instructor interaction and student-student interface.Where appropriate, use plays on words during redundant information presentation.Use humorous introductions.

Humor

Use humorous analogies to explain and summarize.Use creativity techniques to have learners create unusual analogies and associations tothe content.Build in problem solving activities at regular intervals.

Inquiry

Give learners the opportunity to select topics, projects and assignments that appeal totheir curiosity and need to explore.

Participation Use games, role-play, or simulations that require learner participation.

Relevance StrategiesState explicitly how the instruction builds on the learner’s existing skills.Use analogies familiar to the learner from past experience.

Experience

Find out what the learner’s interests are and relate them to the instruction.Present Worth State explicitly the present intrinsic value of learning the content, as distinct from its

value as a link to future goals.State explicitly how the instruction relates to future activities of the learner.Future UsefulnessAsk learners to relate the instruction to their own future goals (future wheel).To enhance achievement striving behavior, provide opportunities to achieve standardsof excellence under conditions of moderate risk.To make instruction responsive to the power motive, provide opportunities forresponsibility, authority, and interpersonal influence.

Need Matching

To satisfy the need for affiliation, establish trust and provide opportunities for no-risk,cooperative interaction.Bring in alumni of the course as enthusiastic guest lecturers.In a self-paced course, use those who finish first as deputy tutors.

Modeling

Model enthusiasm for the subject taught.Provide meaningful alternative methods for accomplishing a goal.ChoiceProvide personal choices for organizing one’s work.

28

ConfidenceStrategies

Incorporate clearly stated, appealing learning goals into instructional materials.Provide self-evaluation tools which are based on clearly stated goals.

LearningRequirements

Explain the criteria for evaluation of performance.Difficulty Organize materials on an increasing level of difficulty; that is, structure the learning

material to provide a “conquerable” challenge.Include statements about the likelihood of success with given amounts of effort andability.Teach students how to develop a plan of work that will result in goal accomplishment.

Expectations

Help students set realistic goals.Attribute student success to effort rather than luck or ease of task when appropriate (i.e.when you know it’s true!).

Attributions

Encourage student efforts to verbalize appropriate attributions for both success andfailures.Allow students opportunity to become increasingly independent in learning andpracticing a skill.Have students learn new skills under low risk conditions, but practice performance ofwell-learned tasks under realistic conditions.

Self-Confidence

Help students understand that the pursuit of excellence does not mean that anythingshort of perfection is failure; learn to feel good about genuine accomplishment.

Satisfactionstrategies

Allow a student to use a newly acquired skill in a realistic setting as soon as possible.Verbally reinforce a student’s intrinsic pride in accomplishing a difficult task.

NaturalConsequences

Allow a student who masters a task to help others who have not yet done so.Reward intrinsically interesting task performance with unexpected, non-contingentrewards.

UnexpectedRewards

Reward boring tasks with extrinsic, anticipated rewards.Give verbal praise for successful progress of accomplishment.Give personal attention to students.Provide informative, helpful feedback when it is immediately useful.

Positive Outcomes

Provide motivating feedback (praise) immediately following task performance.Avoid the use of threats as a means of obtaining task performance.Avoid surveillance (as opposed to positive attention).

Negative Outcomes

Avoid external performance evaluations whenever it is possible to help the studentevaluate his or her own work.Provide frequent reinforcements when a student is learning a new task.Provide intermittent reinforcement as a student becomes more competent at a task.

Scheduling

Vary the schedule of reinforcements in terms of both interval and quantity.

Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journalof Instructional Development, 10 (3), 2-10.

Appendix 2 was generated as an electronic instructional handout for a graduate class inEducational Evaluation offered by the Teaching and Learning Department of the College ofHuman Resources and Education at Virginia Tech located in Blacksburg, Virgina by Dr.Gregory Sherman, 1998.

29

Technology Selection Criteria Matrix Appendix 3

Implementation:Learning/TeachingCharacteristic

Internet___________________

EPSS (Intranet/Extranet ) Email Online Courses

Satellite(wide band 1-way video)

Land Lines/Fiber Optics(compressed2-way video)

CD-ROM/Laser DisksDVD-ROM(CBT)

Audiotape/Videotape

PrintedMaterial

Access (user) Information dial-up(hardware/softwarecompatibility)

Formalorprescheduled

Informaldial-up

Manualdistribution(hardware/softwareneeded)

Unconstrain-ed(hardwareneeded)

Unconstrain-eddistribution

Audience (size) Asynchronous: unlimitedSynchronous: < 8 remote sites

~6 remotesites (18-20learners persite)

Unlimited Unlimited Unlimited

Cost:Communication

minimal High Low-medium NA NA NA

Cost: Start-up Organizational shared cost High Medium Low-medium

Low Low

Fidelity: Audio Low-medium fidelity sound High fidelitysound

High fidelitysound

Highfidelitysound

High fidelitysound

NA

Fidelity: Video Low-medium fidelity motion Full motion Medium-fullmotion

Highqualitygraphics

Full motion NA

InstructionalAdaptability

Can support learning style Can supportlearning style

Can supportlearning style

Supportsremediallearning

Minimalsupport oflearningstyle

Minimalsupport oflearningstyle

InstructionalMethodology

Student-centered and instructorled

Instructor led Student-centered andinstructor led

Self-paced Self-paced Self-paced

InstructionalStrategy

Information dissemination,Q&A, interactive discussion

Lecture,presentationof info onpolicy/procedures

Role-play,Q&A,interactivediscussion,demonstration

Tutorial,drill &practice,simulations

Presentationofinformation

Presentationofinformation

Interaction:Type/Level

Significant student-student andstudent-instructor

Low student-student andmediumstudent-instructor

Significantstudent-student andstudent-instructor

Mediuminteractionstudent andmaterials

NA NA

LearningOutcome (skills,knowledge orattitude)

SKA Primarilyknowledge-based

SKA SKA NA K

Scheduling:Synch/Asynch

Asynchronous/synchronous Live or pre-recorded

Live NA NA NA

Support ServiceNeeded

Minimal Extensive Medium Minimal Minimal NA

UpdateCapability

Just-in-time Just-in-time Just-in-time Mediumflexibility

Inflexible inflexible

Schreiber, D. A., & Berge, Z. L. (1998). Distance Training: How Innovative Organizations Are UsingTechnology to Maximize Learning and Meet Business Objectives. San Francisco: Jossey-Bass.

30

References

Alessi, S., & Trollip, S. (1991). Computer Based Instruction: Methods and Development.New Jersey: Prentice Hall.

Bloom, B., et al. (1984). Taxonomy of educational objectives, Book 1: CognitiveDomain. White Plains: Longman.

Brush, T. A. (1998). Embedding cooperative learning into the design of integratedlearning systems: rationale and guidelines. Educational Technology Research and Development,46(3), 5-18.

Dick, W., & Carey, L. (1996). The systematic design of instruction. (4 ed.). New York:HarperCollins College Publishers.

Gagne, R., & Driscoll, M. (1998). Essentials of learning for instruction. EnglewoodCliffs, NJ: Prentice Hall.

Keller, J. M. (1987). Development and use of the ARCS model of instructional desgin.Journal of Instructional Development, 10(3), 2-10.

Kilpatrick, D. L. (1998). Evaluating Training Programs. San Francisco: Brrett-KoehlerPublishers, Inc.

Mantyla, K., & Gividen, J. R. (1997). Distance Learning: A Step-By-Step Guide forTrainers. Alexandria, VA: ASTD Publications.

McIsaac, M. S., & Blocher, J. M. (1998). How research in distance education can affectpractice. Educational Media International, 35(1), 43-47.

Merrill, M. D., & Tennison, R. (1994). Teaching concepts: an instructional design.Englewood Cliffs, N.J.: Educational Technology.

Moore, M. G. (1996). Theory of transactional distance. In D. Keegan (Ed.), Principles ofDistance Education (pp. 22-38). New York: Routledge.

Moore, M. G., & Kearsley, G. (1996). Distance Education: A Systems View. Boston:Wadsworth Publishing Company.

Phillips, J. J. (1997). Handbook of Training Evaluation and Measurement Methods.Houston: Gulf Publishing Company.

Robinson, D. G., & Robinson, J. C. (1996). Performance Consulting. San Francisco:Berrett-Koehler Publishers.

Schreiber, D. A., & Berge, Z. L. (1998). Distance Training: How InnovativeOrganizations Are Using Technology to Maximize Learning and Meet Business Objectives. SanFrancisco: Jossey-Bass.

Tessmer, M. (1998). Planning and conducting formative evaluations. Philadelphia:Kogan Page Limited.

Tessmer, M., & Richey, R. (1997). The role of context in learning and instructionaldesign. Educational Technology Research and Development, 45(2), 85-115.

Wagner, E. (1997). Interactivity: From Agents to Outcomes. New Directions forTeaching and Learning(71), 19-38.

Weston, C., & Cranton, P. A. (1986). Selecting instructional strategies. Journal of HigherEducation, 57(3), 259-288.

31

Whalen, T., & Wright, D. (1999). Methodology for Cost-Benefit Analysis of Web-BasedTele-Learning: Case Study of the Bell Online Institute. The American Journal of DistanceEducation, 13(1), 24-44.

Wolcott, L. L. (1996). Distant, but not distanced: a learner-centered approach to distanceeducation. TechTrends, 41(5), 23-27.