![Page 1: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/1.jpg)
INSTITUTIONAL REPORTING L E O N E N U R B A S A R I , A N U
PA M E L A S A R LY , A C UH I E D I W I L K I N S O N , U S C
![Page 2: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/2.jpg)
Institutional Reporting from the Australian Graduate Survey
Leone NurbasariPlanning and Performance MeasurementJuly 2014
![Page 3: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/3.jpg)
Presentation Overview• Key Performance Indicators• Program (Course) Review reports• Discipline reporting on CEQ/PREQ• Graduate destination reporting• Comments analyses• Other reporting
3
![Page 4: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/4.jpg)
Key Performance Indicators• ANU Strategic Plan• College Operational Plans and KPIs
4
![Page 5: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/5.jpg)
KPI generation• Raw unit record data at 6 digit FOE, assigned to College scores based on
taught disciplines (rolling 5 year average, minimum 10% of load taught by that College)
• Benchmarking against Go8 data that is weighted according to ANU discipline weightings
5
![Page 6: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/6.jpg)
6
Example of some underlying 6 digit FOE data and calculations for KPI generation
![Page 7: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/7.jpg)
Program (Course) Review Reports• Program reviews • Data warehouse
reports – CEQ, Employment & Further study
• Supplemented by additional detailed analyses (including item level results)
• Supplemented by anonymised open-ended comments
7
![Page 8: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/8.jpg)
Discipline reporting – CEQ/PREQ• Pseudo colleges (custom groupings of 2, 4 and 6 digit FOEs)• Pseudo departments (also custom groupings of FOE)• Benchmarking – Go8 and national, based on ANU taught FOE (excludes non
ANU disciplines)• Percentage of individual student-item response in
disagreement/neutral/agreement categories, aggregated to scales
8
![Page 9: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/9.jpg)
9
![Page 10: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/10.jpg)
Graduate destination reporting• Data warehouse
reports – employment rates, sector, industry, salary, employer, job search
• Cohort-specific analysis
• Ad-hoc data requests
10
![Page 11: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/11.jpg)
Comment analyses• CEQuery with charting via Excel• Various student cohorts and org. units• Adopting Geoff Scott’s method of proxies for
importance and quality
11
![Page 12: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/12.jpg)
Other reporting• Topic Papers – comprising data
from various surveys and student feedback sources, eg. HDR Satisfaction Report
• Ad-hoc reporting from internal pivot tables – GDS, CEQ, PREQ
12
![Page 14: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/14.jpg)
AGS Data and Reporting at ACU
Pamela Sarly | Acting Manager, Statistical Analysis & Surveys
2014 Survey Manager Information Forum17 July 2014
![Page 15: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/15.jpg)
A bit about ACU…
1900
1963
1991Good
SamaritanSisters
Teacher Training in NSW, VIC, QLD and
ACT
Merger of education colleges, forming
ACU 2014
One of the fastest
growing universities in
Australia
1857
TeacherTraining
in NSW and VIC
ONE OF THE FIRST
EDUCATION PROVIDERS IN
AUSTRALIA
![Page 16: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/16.jpg)
Data as at 11 July 2014
29,513 Enrolments22,245 EFTSL2,839 international students
12 Schools across 4 Faculties
12,952 commencing students16,561 continuing students
Staff FTE• 1,051 Academic• 982 Professional
490 Higher Degree Research Students6,263 Postgraduate Students21,934 Undergraduate Students826 Non-Award Students
Australia's leading Catholic university which is supported by
more than 2,000 years of Catholic intellectual tradition.
![Page 17: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/17.jpg)
AUSTRALIAN GRADUATE SURVEYData Governance at ACU
• A central contact for surveys: o Office of Planning and Strategic Management
• Publish aggregated data through internal sites:o Staff Site (www.acu.edu.au/opsm)o SharePoint site
• Data updates communicated to relevant staff via email
![Page 18: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/18.jpg)
![Page 19: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/19.jpg)
AUSTRALIAN GRADUATE SURVEYInstitutional Reporting
• Strategic Plan - Traffic Light Report
• Quarterly Report
• TEQSA Risk Indicators
• New Course Application
• Course Review and Renewal
![Page 20: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/20.jpg)
• Published in July and December
• Progress against the University’s Strategic Goals and Key Result Areas
TRAFFIC LIGHT REPORT
![Page 21: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/21.jpg)
STRATEGIC SCORECARD
![Page 22: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/22.jpg)
• Published in January, April, July and October
• High level quantitative data that is linked to Strategic Goal 1 (Student Experience)
QUARTERLY REPORTStrategic Plan Goal 1: Student Experience
![Page 23: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/23.jpg)
• Use of SPSS TAS for qualitative data analysis and reporting
• Better linkage to Student Evaluation on Unit and Teaching surveys
• Future of AGS – to inform our new Strategic Plan 2015+
AUSTRALIAN GRADUATE SURVEYFor the future…
![Page 24: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/24.jpg)
Thank youQuestions?
![Page 25: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/25.jpg)
COMMUNICATING RESULTS FOR EFFECTIVE DECISION
MAKING
Australian Graduate Survey
Note that the figures contained within this presentation are fictitious.
![Page 26: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/26.jpg)
Why do we need to communicate results?
AGS results are used at USC to:Inform strategic decision makingAssess program and institution performanceInfluence improvements to programsReview learning and teachingDevelop internal policies
![Page 27: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/27.jpg)
Why use visualisations?
“Numbers have an important story to tell. They rely on you to give them a clear and convincing voice.”
Stephen Few
![Page 28: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/28.jpg)
Choosing the right medium
Textual
Visual
Oral
![Page 29: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/29.jpg)
Tapping into human nature
Over 50% of the cerebral cortex is involved with the processing of visual inputs.
-Lu & Dosher
There are over 130 million sight receiving cells in the retina.
- Vries 2011Source: Atranik 2011
![Page 30: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/30.jpg)
Tableau
![Page 31: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/31.jpg)
Knowing your audience
USC’s Organisational Chart
![Page 32: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/32.jpg)
Know your goal
Enable insight
"The purpose of visualization is insight, not pictures"
Shneiderman 1999
![Page 33: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/33.jpg)
GDS Dashboards Flowchart
![Page 34: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/34.jpg)
CEQ Dashboards Flowchart
![Page 35: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/35.jpg)
Levels of Analysis
Graduate Destination Survey Benchmark Group Dashboard
Course Experience Questionnaire Longitudinal Dashboard
Graduate Destination Survey Faculty Dashboard
Graduate Destination Survey Program (Course) Dashboard
Course Experience Questionnaire Program (Course) Dashboard
![Page 36: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/36.jpg)
How dashboards are used
Are they being used?
Who is using them?
What are they using them for?
Director - Quick reference point
Program leaders - Program reviews
Executive - Marketing and recruitment - Identifying strengths and
weaknesses - Information sharing
![Page 37: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/37.jpg)
What next?Reports by Field of
EducationBeyond Graduation SurveyUniversity Experience
SurveyInternal SurveysInteractive DashboardsQualitative Data
Visualisations
![Page 38: INSTITUTIONAL REPORTING LEONE NURBASARI, anu Pamela sarly , acu Hiedi wilkinson , usc](https://reader036.vdocuments.us/reader036/viewer/2022062501/56815c62550346895dca6cfc/html5/thumbnails/38.jpg)
Bibliography
Atranik.org 2011, http://antranik.org/functional-areas-of-the-cerebral-cortex.
Azzam, T & Evergreen, S 2013, Data Visualization, Part 2: New Directions for Evaluation, Number 140, Google eBook.
Card, S, Mackinlay, J & Shneiderman, B 1999, “Readings in Information Visualization - Using Vision to Think”, Morgan Kaufmann, Massachusetts.
De Vries, J 2011, The Five Senses, Random House, Victoria.
Few, S 2006, Information Dashboard Design: The effective visual communication of data, O’Reilly, California.
Lu, Z and Dosher, B 2013, Visual Psychophysics: From Laboratory to Theory, MIT Press, ISBN: 9780262019453.
University of Leicester (Learning Development) 2012, Presenting numerical data, <http://www2.le.ac.uk/offices/ld/resources>