evaluation meets costing: fight or flight? charlotte ash sheffield hallam university 21 st he...
TRANSCRIPT
Evaluation meets costing:Evaluation meets costing:Fight or Flight?Fight or Flight?
Charlotte AshCharlotte AshSheffield Hallam UniversitySheffield Hallam University
http://www.shu.ac.uk/virtual_campus/cnl/team/cea.htmhttp://www.shu.ac.uk/virtual_campus/cnl/team/cea.htm
2121stst HE Cost-effectiveness framework HE Cost-effectiveness frameworkC
The Costs of Networked LearningThe Costs of Networked Learning
Six month study funded by the JISCSix month study funded by the JISC Additional support provided by Sheffield Additional support provided by Sheffield
Hallam UniversityHallam University Project team - director, manager and Project team - director, manager and
additional researchersadditional researchers For further information, visit the Costs For further information, visit the Costs
of Networked Learning Web page: of Networked Learning Web page: http://www.shu.ac.uk/virtual_campus/cnl/http://www.shu.ac.uk/virtual_campus/cnl/
2121stst HE Cost-effectiveness framework HE Cost-effectiveness framework
CostCost
problemsproblems
solutionssolutions
What has been done alreadyWhat has been done already
EffectivenessEffectiveness
problemsproblems
solutionssolutions
C
The Future of Higher EducationThe Future of Higher Education
Universities must:Universities must:
Change infrastructureChange infrastructure
Increase accessIncrease access
Reduce expenditureReduce expenditure
Become Become transparenttransparent
Assessing Cost-EffectivenessAssessing Cost-Effectiveness
Stage 1: Identify problems with costsStage 1: Identify problems with costs
Stage 2: Find solutionsStage 2: Find solutions
Stage 3: Testing and refinementStage 3: Testing and refinement
Stage 4: Identify effectiveness problemsStage 4: Identify effectiveness problems
Stage 5: Find solutionsStage 5: Find solutions
Stage 6: Testing and refinementStage 6: Testing and refinement
Stage 7: Joint solution toStage 7: Joint solution toCost-effectivenessCost-effectiveness
Assessing CostsAssessing Costs
ProblemsProblems
1 Lack of literature1 Lack of literature
2 What to cost?2 What to cost?
3 How to cost?3 How to cost?
4 Whom to cost?4 Whom to cost?
Assessing Costs: Problem 1Assessing Costs: Problem 1
ProblemProblem
Lack of literatureLack of literature
Lack of studiesLack of studies
Lack of frameworkLack of framework
SolutionSolution
Eclectic and Holistic Eclectic and Holistic
approachapproach
Assessing Costs: Problem 2Assessing Costs: Problem 2
ProblemProblem
WhatWhat to cost? to cost?
SolutionSolution
Identify Identify hidden hidden costscosts
Formulate list for Formulate list for
each stakeholder: each stakeholder:
staff, student and staff, student and
institutioninstitution
Structure list into Structure list into
working modelworking model
Lifecycle modelsLifecycle models
Planning the learning experience/ environment
Maintaining the learning experience/ environment
Developingthe learning experience/ environment
Managing the learning experience/ environment
Providing the learning experience/ environment
Academic Staff
Support Staff
Students
Planning and Development
Productionand
Delivery
Maintenanceand
Evaluation
Final three stage model
Previous five stage model
Expendituredimension
Stakeholder dimension Total
Institution Student Staff
Staff costs
Depreciation
Expenses
Overhead
Total
Assessing Costs: Problem 3Assessing Costs: Problem 3
ProblemProblem
HowHow to cost? to cost?
SolutionSolution
Planning frameworkPlanning framework
• Planning documentPlanning document
• Financial schemaFinancial schemaFinancial SchemaFinancial Schema
Assessing Costs: Problem 4Assessing Costs: Problem 4
ProblemProblem
WhomWhom to cost? to cost?
• Whom to include?Whom to include?
• Who to do?Who to do?
SolutionSolution
• StakeholdersStakeholders
• Translator/NegotiatorTranslator/Negotiator
SolutionsSolutions
1 Eclectic/Holistic approach1 Eclectic/Holistic approach
2 Hidden, List2 Hidden, List ModelModel
3 Planning framework3 Planning framework
• planning documentplanning document
• financial schemafinancial schema
4 Whom to cost:4 Whom to cost:
• StakeholdersStakeholders
• Translator/NegotiatorTranslator/Negotiator
Assessing CostsAssessing Costs
ProblemsProblems
1 Lack of literature1 Lack of literature
2 2 WhatWhat to cost? to cost?
3 3 HowHow to cost? to cost?
4 4 WhomWhom to cost? to cost?
• Whom to include?Whom to include?
• Who to do it?Who to do it?
Assessing EffectivenessAssessing Effectiveness
Reflections on a Model forReflections on a Model forEvaluating Learning TechnologiesEvaluating Learning Technologies
by Jones, Scanlon and Blakeby Jones, Scanlon and Blake
CIAO! evaluation frameworkCIAO! evaluation framework developed by the Computers and Learning developed by the Computers and Learning
Research Group at the Open UniversityResearch Group at the Open University
““Do my students learn better, fasterDo my students learn better, fasteror more using this technology?”or more using this technology?”
Assessing EffectivenessAssessing Effectiveness
EvaluatingEvaluatingTechnology-Based LearningTechnology-Based Learning
bybyDr Robin MasonDr Robin Mason
Holistic evaluationHolistic evaluation Good evaluations demandGood evaluations demand
skill, perception and resourcesskill, perception and resources
Flashlight 1994 - 1999Flashlight 1994 - 1999
Current Student Inventory Current Student Inventory ((19941994))
• effectivenesseffectiveness
Costs Analysis Handbook Costs Analysis Handbook ((19991999))
• costscosts
Problems:Problems:1 Costs and effectiveness kept separate1 Costs and effectiveness kept separate
2 Individualistic approach prohibits comparisons2 Individualistic approach prohibits comparisons
Assessing Cost-EffectivenessAssessing Cost-Effectiveness
Stage 1: Identify problems with costsStage 1: Identify problems with costs
Stage 2: Find solutionsStage 2: Find solutions
Stage 3: Testing and refinementStage 3: Testing and refinement
Stage 4: Identify effectiveness problemsStage 4: Identify effectiveness problems
Stage 5: Find solutionsStage 5: Find solutions
Stage 6: Testing and refinementStage 6: Testing and refinement
Stage 7: Stage 7: Joint solution Joint solution to problem ofto problem of
evaluating the cost-effectiveness of HEevaluating the cost-effectiveness of HE
Evaluation meets costing:Evaluation meets costing:Fight or Flight?Fight or Flight?
Charlotte AshCharlotte AshSheffield Hallam UniversitySheffield Hallam University
http://www.shu.ac.uk/virtual_campus/cnl/http://www.shu.ac.uk/virtual_campus/cnl/
Thank you for listeningThank you for listening