current state and trends in learning analytics developments in la_1.pdf · current state and trends...

Current State and Trends in Learning Analytics

Dragan Gašević

@dgasevic

January 22, 2016 Open University of Hong Kong Hong Hong

Educational Landscape Today

“Non-traditional” students

Redefining the role of universities

Changing labor market

Learning at scale

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 201319030.

Feedback loops between students and instructors

are missing/weak!

LEARNING ANALYTICS

Learning environment

Educators

Learners Student

Information Systems

Blogs

Videos/slides

Mobile

Search

Educators

Learners

Networks

Student Information

Systems


Blogs

Mobile

Search

Networks

Educators

Learners Student

Information Systems


Videos/slides

Learning Analytics – What?

Measurement, collection, analysis, and reporting of data about

learners and their contexts

Learning Analytics – Why?

Understanding and optimising learning and the environments

in which learning occurs

Pass/Fail, Retention

Concept understanding

Learning motivation/engagement

Learning strategy and metacognition

Learning dispositions

Graduate qualities

Learning experience

Satisfaction, community

Questions explored

Growing Interest

https://campustechnology.com/articles/2015/12/01/blackboard-acquires-predictive-analytics-company-blue-canary.aspx

https://thejournal.com/articles/2015/07/02/blackboard-acquires-xray-analytics-for-moodle.aspx

http://www.prweb.com/releases/2015/07/prweb12854557.htm

CASE STUDIES

Student retention

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Year 1 Year 2 Year 3 Year 4

Course Signals

No Course Signals

Arnold, K. E., & Pistilli, M. D. (2012, April). Course Signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 267-270).

Tanes, Z., Arnold, K. E., King, A. S., & Remnet, M. A. (2011). Using Signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4), 2414-2422.

Can teaching be improved?

http://quantifiedself.com/about/

Patel, M. S., Asch, D. A., & Volpp, K. G. (2015). Wearable devices as facilitators, not drivers, of health behavior change. The Journal of the American Medical Association, 313(5), 459-460.

Wright, M. C., McKay, T., Hershock, C., Miller, K., & Tritz, J. (2014). Better Than Expected: Using Learning Analytics to Promote Student Success in Gateway Science. Change: The Magazine of Higher Learning, 46(1), 28-34.

INSTITUTIONAL ADOPTION: CURRENT STATE

Very few institution-wide examples of adoption

Stage 1: Extraction and reporting of transaction-level data Stage 2: Analysis and monitoring of operational performance Stage 3: “What-if” decision support (such as scenario building) Stage 4: Predictive modeling & simulation Stage 5: Automatic triggers and alerts (interventions)

Goldstein, P. J., & Katz, R. N. (2005). Academic analytics: The uses of management information and technology in higher education (Vol. 8). Educause.

Analytics Framework

~70% institutions in phase 1 305 institutions, 58% at stage 1, 20% at stage 2

Yanosky, R. (2009). Institutional data management in higher education. ECAR, EDUCAUSE Center for Applied Research.

Goldstein, P. J., & Katz, R. N. (2005). Academic analytics: The uses of management information and technology in higher education (Vol. 8). Educause.

Interest in analytics is high, but many institutions had yet to make progress beyond basic reporting

Bichsel, J. (2012). Analytics in higher education: Benefits, barriers, progress, and recommendations. EDUCAUSE Center for Applied Research.

Sophistication model

Siemens, G., Dawson, S., & Lynch, G. (2014). Improving the Quality and Productivity of the Higher Education Sector - Policy and Strategy for Systems-Level Deployment of Learning Analytics. Canberra, Australia: Office of Learning and Teaching, Australian Government. Retrieved from http://solaresearch.org/Policy_Strategy_Analytics.pdf

Current state

Benchmarking learning analytics status, policy and practices for Australian universities

Senior management perspective

Solution-driven approach

Bought an analytics product.

Analytics box ticked!

Lack of data-informed decision making culture

Macfadyen, L., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan. Educational Technology & Society, 15(3), 149-163.

Researchers not focused on scalability

Learning from the successful examples

An institutional learning analytics vision

Tynan, B. & Buckingham Shum, S. (2013). Designing Systemic Learning Analytics at the Open University. SoLAR Open Symposium – Strategy & Policy for Systemic Learning Analytics. http://people.kmi.open.ac.uk/sbs/2013/10/designing-systemic-analytics-at-the-open-university

What’s necessary to move forward?

DIRECTIONS

Data – Model – Transform

Barton, D., & Court, D. (Oct 2012). Making Advanced Analytics Work for You. Harvard Business Review, 79-83, https://hbr.org/2012/10/making-advanced-analytics-work-for-you/ar/1

https://hbr.org/2012/10/making-advanced-analytics-work-for-you/ar/1












Creative data sourcing













Social networks are everywhere

Gašević, D., Zouaq, A., Jenzen, R. (2013). ‘Choose your Classmates, your GPA is at Stake!’ The Association of Cross-Class Social Ties and Academic Performance. American Behavioral Scientist, 57(10), 1459–1478.


Creative data sourcing

Necessary IT support













Awareness of limitations and challenging assumptions

Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. (in press). Does Time-on-task Estimation Matter? Implications on Validity of Learning Analytics Findings. Journal of Learning Analytics, 2(3).


Question-driven, not data-driven













Learning analytics is about learning

Gašević, D., Dawson, S., Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.

Once size fits all does not work in learning analytics

Gašević, D., Dawson, S., Rogers, T., Gašević, D. (2016). Learning analytics should not promote one size fits all: The effects of course-specific technology use in predicting academic success. The Internet and Higher Education, 28, 68–84.

Learning context

Instructional conditions shape learning analytics results

Learner agency

Kovanović, V., Gašević, D., Joksimović, S., Hatala, M., Adesope, S. (2015). Analytics of Communities of Inquiry: Effects of Learning Technology Use on Cognitive Presence in Asynchronous Online Discussions. The Internet and Higher Education, 27, 74–89.

More time online does not always mean better learning


Participatory design of analytics tools















Analytics tools for non-statistics experts













Visualizations can be harmful

Corrin, L., & de Barba, P. (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In Proceedings of the ascilite 2014 conference (pp. 629-633). ascilite.



Analytics tools for non-statistics experts

Develop capabilities to exploit (big) data Barton, D., & Court, D. (Oct 2012). Making Advanced Analytics Work for You. Harvard Business Review, 79-83, https://hbr.org/2012/10/making-advanced-analytics-work-for-you/ar/1












Marr, B. (Oct 2015). Forget Data Scientists - Make Everyone Data Savvy, http://www.datasciencecentral.com/m/blogpost?id=6448529%3ABlogPost%3A337288

http://www.datasciencecentral.com/m/blogpost?id=6448529:BlogPost:337288

http://www.datasciencecentral.com/m/blogpost?id=6448529:BlogPost:337288

Are we ready to act on analytics?

What to do if we detect deficit models in our practice?


Joksimović, S., Gašević, D., Loughin, T. M., Kovanović, V., Hatala, M. (2015). Learning at distance: Effects of interaction traces on academic achievement. Computers & Education, 87, 204–217.

How do we deal with performance-oriented culture?


Jovanović, J., Pardo, A., Gašević, D., Dawson, S., Mirriahi, N. (2015). Dynamic analytics of learning in flipped classrooms. Manuscript in preparation.

LA idealized systems model

Colvin, C., Rogers, T., Wade, A., Dawson, S., Gasevic, D., Buckingham Shum, S., Nelson, K., Alexander, S., Lockyer, L., Kennedy, G., Corrin, L., Fisher, J. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Australian Goverement’s Office for Learning and Teaching.

Macfadyen, L. P., Dawson, S., Pardo, A., & Gasevic, D. (2014). Embracing big data in complex educational systems: The learning analytics imperative and the policy challenge. Research & Practice in Assessment, 9(2), 17-28.

Rapid Outcome Mapping Approach (ROMA)

FINAL REMARKS

Embracing complexity of educational systems

Capacity development

Multidisciplinary teams in institutions critical

Ethical and privacy consideration Development of data privacy agency

Prinsloo, P., & Slade, S. (2015). Student privacy self-management: implications for learning analytics. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 83-92). ACM.

Sclater, N. (2014). Code of practice for learning analytics: A literature review of the ethical and legal issues. http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf

Development of analytics culture

Manyika, J. et al. (2011). Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute, http://goo.gl/Lue3qs

Many thanks!

current state and trends in learning analytics developments in la_1.pdf · current state and trends...

Documents