Meeting Technology Literacy Expectations: Assessing and

Assisting Incoming StudentsCathy Manly Director of Distance Learning

Manchester Community College

Connecticut Distance Learning Consortium

NERCOMP Conference, March 2005

Copyright Cathy Manly 2005. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that

this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written

permission from the author.

Why Technology Literacy?

• Students without basic skills struggle with technology instead of academic content

• Some students drop courses because of technology requirements

• Online students need technology skills, but increasingly classroom students need them also

Addressing Tech Lit

• Survey• Identify what constitutes basic tech lit

• Assessment• Identify how to assess these skills

• Remediation• Identify possible remediation alternatives

Formation of CTDLC--1998

WWW.CTDLC.ORGMembers: 37 Higher Ed; 10 Affiliates

CTDLC-FIPSE Collaborators

•Private 4-year institutions:Quinnipiac University Sacred Heart University

Post University

•Public 4-year institutions:Charter Oak St. College Univ. of Connecticut

•Public 2-year institutions:Capital CC Manchester CCNaugatuck Valley CC Northwestern CT CCThree Rivers CC Tunxis CC

CTDLC’s eSuite

Funded by Grants: Fund for the Improvement of Post Secondary Education and Davis Education Foundation

Building eSuite Components

Tech Lit Assessment Directions

• Cover faculty-identified items• Easy to administer and get results

• Institutional identification

• Accessible online• Offer remediation possibilities

• Basic remediation (through the grant)• Indicate institutional assistance

Tech Skill Survey Results

• Fall 2002 survey• 541 faculty and staff at 11 institutions

• What tech skills should students have on the first day of class?• Choices: Required, Helpful, Not Required

• Targeted “required” skills • Identified by > 50% of faculty teaching

introductory courses

Faculty Expectations of Student Technology Literacy

• Basic computer use• Basic word processing• Basic email• Basic web use

Basic Computer Use

• Use a mouse (80, 94%)• Type/keyboard (79, 96%)• Use copy and paste (54, 85%)• Access a CD-ROM (52, 76%)• Open and exit a software application

In parentheses: percent of faculty teaching introductory courses indicating the skill is: (required, required+helpful.)

Basic Word Processing

• Use a word processing program (76, 91%) to create text reports or papers (72, 87%)• Create a blank line• Tab text to indent a line• Apply basic formatting (bold, italics, center)

• Save a file (71, 88%)• Locate a saved file (70, 88%)• Print a file (69, 88%)

Basic Email

• Send and receive email (73, 93%)• Attach a file to an email (60, 88%)

Basic Web Use

• Go to a specified website (79, 93%)• Use an internet browser (78, 95%)

• Click on a link• Scroll through a webpage• Use the Back button

• Use a search engine (64, 92%)• Perform a basic search and identify search results

• Download and view a file from a webpage (57, 83%)• Print a webpage (58, 85%)

Highest Expectations

• Use a mouse (80, 94%)• Type/keyboard (79, 96%)• Use a word processing program (76, 91%)• Send and receive email (73, 93%)• Go to a specified website (79, 93%)• Use an internet browser (78, 95%)

3 Assessment Methods Compared

• Method 1: Self-assessment• CTDLC created• Students answer 1-4 or n/a

• Method 2: Tasks• CTDLC created• Web form-based assignments

• Method 3: Simulation• SAM by Course Technology• Test environment captures mouse and typing

Piloting the Assessments

• First Pilot – January 2004• 403 students, 8 schools - self-test and web

assignment• 360 students, 2 schools - SAM simulation

• Second Pilot – April/May 2004• 643 students from 9 schools

• 169 students - self-test• 474 students - web assignment

• 215 students, 1 school - SAM simulation

Method 1: Self-Assessment

• Insignificant correlation between self-assessment and simulation

• Most students indicated they could “do easily” things they were unable to do

• Our conclusions:• Useful for student self-knowledge • Useful to direct students to assistance• Not an accurate indicator of student skill

Method 2: Tasks

• Learned from both pilots• Revised web piece for 2nd pilot• Now revising word processing part

• 20 min timeframe provides good results • Average <10 min, range 3-47 min• Placement testing situations benefit from

enforced time limits

• Observation showed some students clearly struggled with skills

Method 3: Simulation (SAM)

• SAM simulation worked well• 20 min timed test; students averaged 9 min

• Observation showed it accurately measures student ability

• Over 40% of students need assistanceStudent scores from the first pilot:• 11% need significant help (F)• 31% need review (C-D)• 58% are okay (A-B)

Demonstrated Student Tech Skills and Deficiencies

• An unexpectedly high number of students need significant help or review

• Students were best at:• Basic computer operation, mouse• Using links on a webpage, Back button

• Students had the most difficulty with:• Email (send/receive/attachments)• Going to a specific web address• Copy and paste

Correlation Investigation

• No significant detectable correlation found between simulation scores and:• Age• Gender• Race• Full-time versus part-time• Length of time since high school

Analysis done with Manchester CC students

Relationship Between Tech Skills and Placement Scores

• Significant relationship found between simulation scores and both:• English placement test scores • Math placement test scores

• Students with better English or math placement test scores were more likely to do well on the tech lit assessment

Analysis done with Manchester CC students

Possible Remediation Alternatives Investigated

• Incorporation into regular classes• Credit or non-credit tech courses• Workshops specifically covering basic

tech skills• Online remediation modules currently

under development

Online Remediation Modules

Coming for our Spring 2005 pilot

Basic Computer Skills Remediation• Self-Paced• Available in a variety of LMS• Designed for these very basic skills

Future: Spring 2005 Pilot

• Third Pilot – April/May 2005• Focus on all incoming students taking

placement testing• Only using SAM Challenge simulation

• Goals• Are our results repeatable?• Expand our correlation investigation

To View a Demo

• Interested in viewing a demo of the technology literacy assessment?• Go to http://www.ctdlc.org/techlittest/

(This link is on the handout.)

Q&A

• Cathy Manly, Manchester CCcmanly@mcc.commnet.edu

• More information about the project:http://www.ctdlc.org/Evaluation/grants-FIPSE.html• Contact Diane Goldsmith, CTDLC

dgoldsmith@ctdlc.org