the academic computing assessment data repository: a new (free) tool for program assessment heather...

The Academic Computing Assessment Data Repository: A New (Free) Tool for Program Assessment

Heather Stewart, Director, Institute for Technology Development, Seton Hall University

Dr. Bert Wachsmuth, Chair, Dept. of Mathematics and Computer Science, Seton Hall University

Dr. Eric Fountain, Research Associate, Institute for Technology Development, Seton Hall University

Copyright Heather Stewart, Bert Wachsmuth, Eric Fountain 2001. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.





A New Tool for Assessment• Heather Stewart: The Assessment of Large-Scale Technology Initiatives

• Eric Fountain: ITD and ACADR

• Bert Wachsmuth: A New Online Survey Design and Administration Tool

Charge of the Mobile Assessment Team

• Assess the impact of mobile computing on fall 1998 entering freshman

• This includes student

satisfaction with the learning environment and student learning outcomes

Narrowing the Question

Fantasy:a + b = c

a = teaching, b = technology, c = learning

Reality:a + b + c + d + e + f…

discipline, teaching style, technology proficiency, fit of application to task,

desired learning objectives...

When Assessment Occurs

During

Make Changes in Process

Before

Determine Needs

After

Evaluate Experience

Examples of Assessment Topics

Faculty Students Institution

•Technical Skills•Instructional Goals

•Technical Skills•Enrollment Decisions

•Community Engagement•Resources

•Support•Time Distribution

•Access• General Satisfaction

•Bottlenecks•Network Stability

•Training Needs•Tool Fits Goal

•Info. Access•Communication•Technical Skills

•Delivery Change•Evaluate Tech. Options

D

A

B

Types of Assessment

• Focus Groups• Interviews• Journals• Observations• Portfolios• Performances

• Standardized Tests• Surveys• Tracking Systems• Longitudinal Records• Document Reviews• Skills Application

Demonstrations


ITD and ACADR


ACADR Vision •Resource for Assessment

•Qualitative and Quantitative Data

•Best Practice Information

•Research and Assessment Reports

Survey Tool – First Offering


• What will it do?

•Conditions of Use

•Shared Resource

•Data – Across Institutions and Over Time

ACADR

ACADR: TechoTalk•Overview

•Taking a Survey

•Downloading/View Data

•Administrating Surveys

•The Inside View

•Participate or Download

Eric Fountain & Bert Wachsmuth

ACADR

Overview

•ACADR: Academic Computing Assessment Data Repository

•Collect, evaluate, and share technology assessment data

•General tool for survey administration and evaluation

•Three Web-based Modules

•Submission module for users to take a survey

•Download/View module for researchers to evaluate data

•Administration module to create and manage surveys

•Free Availability

•Use Seton Hall server for technology assessment

•Install on your own server for general use


ACADR

Taking a Survey

•Pick a survey or jump via direct link

•Password-protected surveys

•Customized survey appearance

•Captures user and survey-specific information

•Recognizes returning users

Planned Improvements:

•Authenticate against LDAP server

•Restrict survey to selected users


ACADR

Download/View Data

•Available to ‘registered researchers’ only

•User names available to ‘trusted’ researchers only

•View online descriptive summary

•Combine data across surveys for cross-referencing

•Download data in Excel or SPSS format

Planned Improvements:

•Expand cross-referencing capabilities

•Add additional descriptives to online summary


ACADR

Administrating Surveys

•Available to ‘registered administrators’ only

•Support for multiple preconfigured ‘campuses’

•Create surveys, factors, metrics, etc. online

•Define valid date range and password per survey

•Reuse and customize surveys via ‘instances’

•An ‘instances’ customizes a survey for a campus

•A ‘survey’ consists of one or more groups

•A ‘group’ consists of factors and metric groups

•Tracks editing changes


ACADR

The Inside View

•Uses secure, modern technologies

•Linux Operating System

•Apache Web Server

•IBM DB2 SQL server

•Java Servlets

•Uses Open Standards Technology

•‘Frontend’ works with any Java Servlet 1.0 web server

•‘Backend’ works with any JDBC 2.0 SQL server

Planned Improvements

•Move to SSL-secured web site


ACADR

Participate or Download

•For Technology Assessment in Higher Education

•Manage surveys through SHU server w/o setup or cost

•Benefit from shared surveys, factors, and metrics

•Enhance all IT assessment via cross-referencing

•For General Use

•Download and install software (GNU-like license)

•Use for institutional assessment (e.g. faculty evaluations)

•Support faculty research needs

•Available for IT Assessment Pilots now

•Available for (registered) download Jan. 15, 2002



An Invitationhttp://repository.itd.shu.edu

[email protected]




the academic computing assessment data repository: a new (free) tool for program assessment heather...

Documents

assessment qualitative

assessment reports

program assessment itd

new tool

acadr slide

b slide

new free tool

mobile assessment team