capturing learning paper
TRANSCRIPT
-
7/28/2019 Capturing Learning Paper
1/48
Abstract
This pilot study employs a design-based research methodology (Van den Akker, 2006) and
critical realist analytical framework (Danermarket al., 1997), to evaluate the design and
implementation of an online learning space, for a group of approximately 60 initial teacher
education students in their final year.
The inquiry aims to describe how learning happened in the online space, and explain how the
design of the online space influenced learning. Five contextually-sensitive design principles are
hypothesised, and these are interrogated through the lens of Mezirows transformative learning
theory (2009).
The design principles hypothesise that pubic yet safe online spaces have the potential to foster
student voice and the performance of professional identities. Is is suggested that transformative
learning has the potential to emerge if the technology affords an opportunity for reflexive
engagement with this re-performance of identity.
A conflict between the realist ontology and constructivist epistemology of the critical realism
meta-framework is highlighted as a major challenge for educational research, as well as the
complex nature of inquiry into socio-technological learning. The value of complexity theory
Contents
Abstract
Preface: digital scholarship
Introduction
Co ntext: educ ational turbulence
Context: online learning spaces
Appro ach: critical realist hacking
Approach: design-based research
Approach: integrated methods
Approach: critical realist analysis
Analysis: des cribing the des ign
Analysis: describing learning
Analysis: e xplaining learning
Analysis: theorising learning
Recontextualisation
Looking bac k: limitations
Looking forward: possibilities
Postscript: musings on authority
Search
CAPTURING LEARNING
Home About References MATHAGOGY PEPS MCCREA
PDFmyURL.com
http://pepsmccrea.com/capturing-learning-paper/#11http://pepsmccrea.com/capturing-learning-paper/#9http://pepsmccrea.com/capturing-learning-paper/#7http://pepsmccrea.com/capturing-learning-paper/#2http://pepsmccrea.com/capturing-learning-paper/#2http://www.pepsmccrea.com/capturing-learning-paper/http://www.pepsmccrea.com/capturing-learning-paper/?page_id=526http://www.pepsmccrea.com/capturing-learning-paper/http://www.pepsmccrea.com/capturing-learning-paper/http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://creativecommons.org/licenses/by/3.0/http://pepsmccrea.com/capturing-learning-paper/#17http://pepsmccrea.com/capturing-learning-paper/#16http://pepsmccrea.com/capturing-learning-paper/#15http://pepsmccrea.com/capturing-learning-paper/#14http://pepsmccrea.com/capturing-learning-paper/#13http://pepsmccrea.com/capturing-learning-paper/#12http://pepsmccrea.com/capturing-learning-paper/#11http://pepsmccrea.com/capturing-learning-paper/#10http://pepsmccrea.com/capturing-learning-paper/#9http://pepsmccrea.com/capturing-learning-paper/#8http://pepsmccrea.com/capturing-learning-paper/#7http://pepsmccrea.com/capturing-learning-paper/#6http://pepsmccrea.com/capturing-learning-paper/#5http://pepsmccrea.com/capturing-learning-paper/#4http://pepsmccrea.com/capturing-learning-paper/#3http://pepsmccrea.com/capturing-learning-paper/#2http://pepsmccrea.com/capturing-learning-paper/#1http://pepsmccrea.com/http://www.pepsmccrea.com/capturing-learning-paper/?page_id=1084http://www.pepsmccrea.com/capturing-learning-paper/?page_id=70http://www.pepsmccrea.com/capturing-learning-paper/?page_id=526http://www.pepsmccrea.com/capturing-learning-paper/http://www.pepsmccrea.com/capturing-learning-paper/ -
7/28/2019 Capturing Learning Paper
2/48
(Davis & Sumara, 2008), and the hacker ethos (Suiter, 2010) is explored in this context.
This study attempts to rise the the responsibilities of scholarship in a digital age, and has been
conducted using digital, networked and open practices where appropriate (Weller, 2012).
The online version of this report can be found at http://www.pepsmccrea.com/capturing-learning-
paper/ and a pdf version of the main body of the report can be accessed by clicking here.
Preface: digital scholarship
This is a report about my research into learning and design for learning in a digital age.
However, it also represents my developing understanding of what it means to be a scholar in these
times, and in particular the ethical choices facing contemporary researchers. In light of this, I have
chosen to approach both the doing of the research and the reporting of the research using the
following digital, networked and open practices:
I have been using Twitter to connect to networks of expertise, providing access to highly
current and relevant content, and as a way to share my thinking with others
I have been blogging about my research, which has encouraged me to reflect on and articulate
my thinking, and to create a space for discussion and feedback on my ideas
I have analysed data using tools that allow them to be represented digitally and shared openly
(after they have been anonymised) for reuse and scrutiny in meta-analyses and parallel studies
I am publishing this report openly on the internet with a Creative Commons license so anyone
(with the internet) can free ly access and build upon my work
I have built this report as a hyperlinked document to offer readers a connected experience,
where they can be transported directly to source material
I have designed this report so that readers can comment on it if they wish
I have linked this report to my digital identity so readers can easily evaluate my credibility as
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/fki3http://www.pepsmccrea.com/capturing-learning/wp-content/uploads/2012/03/www-capturinglearning-com-3.pdfhttp://www.pepsmccrea.com/capturing-learning-paper/ -
7/28/2019 Capturing Learning Paper
3/48
a researcher, and connect with me if they wish
As you can see from this list, digital scholarship is more than just using technologies for effective
collaboration and research it is as much about embracing open values and exploiting the
potential o f technology for the benefit o f the academy and so ciety (Weller, 2011). This includes:
championing the importance of sharing and being generous (Wiley, 2010); recognising the need
to practice inclusion on a larger scale (Meiszner, 2011); developing new systems
for licensing content; and exploring alternative models for resourcing learning.
The open movement is not new, but it does appear to be gaining momentum. At the time of writing
[21/12/11] the Directory of Open Access Journals had over 7000 periodicals in its
books. Furthermore , increasing numbers of academics are committing themselves to open
practices by signing the Open Access Pledge , and the UK government is developing policy to
encourage the publication of publicly funded research in Open Access Journals (Jha, 2011).
The open revolution also faces some significant challenges as it grows in scale: issues of quality,
identity and ownership (Weller, 2011). How does networked participation impact on
the democratisation of knowledge creation and dissemination (Veletsianos, 2011). Can more open
systems of publication match the academic rigor of established journals? What are the implications
for academics of having a significantly more visible digital footprint? How do these practice fit
with current institutional financial models of intellectual property? And how do we recognise and
ce lebrate intellectual contributions in more open forums?
One innovation that is attempting to address some of the structural issues outlined here is Creative
Commons. Creative Commons provides an free, public and standardised infrastructure and set of
licenses that offer a balance between stringent copyright law and the reality of (looser) internet
publication practices. Fig. 1 desc ribed some of the components of the licences:
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/fki3http://goo.gl/8R0Xrhttp://goo.gl/gaI9vhttp://goo.gl/6h9qxhttp://goo.gl/PqsmOhttp://goo.gl/TtJ1ghttp://goo.gl/gHA7phttp://goo.gl/HWRshttp://goo.gl/gaI9v -
7/28/2019 Capturing Learning Paper
4/48
Fig. 1Creative Commons licence codes (UCT, no date)
www.capturinglearning.com is licensed by me, peps mccrea under a Creative Commons Attribution
3.0 Unported License. This means that anyone is free to copy, adapt and distribute this research, as
long as they attribute it to me. The image below (and on the top right) is a machine readable version
of this license:
Fig. 2Creative Commons licence for www.capturinglearning.com
I am keen to explore the publication of my thesis in a similar manner, and wish to use this pilot as a
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://creativecommons.org/licenses/by/3.0/http://creativecommons.org/licenses/by/3.0/http://goo.gl/yTjEShttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/12/What-is-a-Creative-Commons-license.jpg -
7/28/2019 Capturing Learning Paper
5/48
test case to explore the position of open publishing within our institutional systems. I am not the
first person to do this for an interesting example check out Belshaws (2011) thesis on Digital
Literacies, and for a more detailed coverage of the area see Andrew et al.s (2012) SAGE
Handbook of Digital Dissertations and Theses.
Introduction
This is the report of a pilot project I am undertaking in preparation for my EdD thesis, exploring
blended learning in higher education. It is aimed at helping me understand the ro le and influence of
digital technologies on learning in the context of my practice as a secondary mathematics teacher
educator.
To help me do this I have designed and built a public, online discussion space using freely
available webspace building software (WordPress), and am encouraging the 60 (mostly
pos tgraduate) teacher trainees on my course to digest and comment on artefacts (articles, videos,
websites etc.) that I post each week during the first 9 weeks of their course. The online space is
called MATHAGOGY (link available on header bar at top of the page).
My approach uses a modified design-based research (Van Den Akker et al, 2006) methodology
and critical realist analytical framework (Danermark et al, 1997) to help me design, build and
evaluate an online learning space andanswer the below questions. Design-based research begins
with the premise that learning experiences can andshould be continually improved, and so my
rese arch questions are not built around an existing problem, but rather a potential oppo rtunity:
1. How do learners learn in the online space I have designed and made?
2. How does the design of the online space influence their learning?
My research and thinking about the world is located in a post-positive paradigm best described as
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.capturinglearning.com/?page_id=1084http://wordpress.com/http://goo.gl/pbhu2http://goo.gl/Wkl7b -
7/28/2019 Capturing Learning Paper
6/48
a critical realism (Bhaskar, 1975). As a realist I will be seeking to accurately describe what the
MATHAGOGY learning experience looks like, as well as trying to develop a set of context-
dependent generalisations that explain how the structure of MATHAGOGY and the agency of the
learners using it are related. Critical realists describe these observable and non-observable links
between structure and agency as causal mechanisms.
A significant feature of the research design is an absence of initial conceptual framework. One of
the main purposes of this pilot to to support me in identifying an appropriate lens and set of
questions from which I can launch my thesis. Despite this predominantly inductive approach, I am
aware that I will not be conducting value-free research. This project (like all aspects of my
practice) are heavily influenced by my socio-cultural context, history, and pedagog ical values :
what I believegood learning and teaching looks like. This project provides a rich opportunity to
explicate my tacit assumptions about pedagogy, and develop a more visible understanding of my
pedagog ic practice something that educational des igners have found to be an invaluable by-
product of their engagement with des ign for learning (Swan, 2008 and Beetham & Sharpe, 2007).
This is significant for me in my currentcontext, but will be even more influential in my future. As
an early career academic I am still in the formative stages of developing a theoretically grounded
pedagog ical approach. However, considering that it is likely that I will s till be practicing in 2050
(Osborne, 2011) a secure grounding in digital literacies ( Belshaw, 2011) looks set to be equally
important.
Context: educational turbulence
Higher education in the UK is currently striving to adapt in a turbulently changing economic, social
and technological landscape: an era that is described increasingly as the digital age (JISC, 2009).
The rise of mobile devices and web 2.0 technologies mean that digital practices are becoming a
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/o8wq5http://goo.gl/2pCNghttp://goo.gl/sn4jphttp://goo.gl/X0OJx -
7/28/2019 Capturing Learning Paper
7/48
taken-for-granted part of our daily blended lives, whilst the demand for digital literacies across a
wide range of sectors continues to grow (Horizon, 2011). Blended in this context refers to
mixed use of digital and non-digital tools. For a further exploration of digital literacies, please
refer to Belshaw (2011).
Accelerating technological innovation is not the only major influence on the digital age. Severe
constraints on spending (as in the recent Comprehensive Spending Review, Browne Report, andWhite Paper) are adding to the challenging context for higher education. This situation is forcing
institutions to explore alternative models of education, and technology has been touted as a
solution to cut costs, enhance marketing and meet the rising expectations of students. The UK task
force recently identified blended learning as a real opportunity for UK institutions to develop
responsive, engaging and interactive provision which, if offered at scale, can deliver quality and
cost-effectiveness and meet student demands for flexible learning (HEFCE, 2011).
Student expectations also appear to be changing. Experiences are being sought that are moreconsistent with their blended lives. Learners are increasingly demanding to learn whenever and
wherever they want: in essence, to have a more blended learning experiences ( Johnson, 2011).
However, the ECAR (2010) study suggests that institutions continue to make poor use of digital
technologies (see Fig. 3 below). While course management systems are indeed prevalent, this is
only narrowly exploiting the potential of technology for learning.
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.educause.edu/studentsAndTechnologyInfographichttp://goo.gl/GbuLkhttp://goo.gl/Mgwd9http://goo.gl/Y4nyphttp://goo.gl/bJ8Oshttp://goo.gl/ifmUhttp://goo.gl/GbuLk -
7/28/2019 Capturing Learning Paper
8/48
Fig. 3Infographic of the ECAR ( 2010) national study of students and technology
The promise of technology for education is compelling. However, the uptake of blended learning
in higher education has been slow, and technology has not had the transformative impact on
learning that many had hoped (Conole, 2011). I suggest that one of the reasons for this is the huge
challenge of enabling a digitally literate academic staff body. Developing both a robust
understanding of technology and its role in learning requires sustained engagement, and a
commitment to changing practice .
HEFCEs (200 9) revised strategy for e -learning and JISCs (2009) Effec tive practice in a digital
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/pPnTwhttp://www.educause.edu/studentsAndTechnologyInfographichttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/09/Students-and-Technology-Infographic-EDUCAUSE-e1324713780823.png -
7/28/2019 Capturing Learning Paper
9/48
age made a start on that agenda, challenging institutions to move beyond capital investment in
technology, towards the engagement of staff in the use of technologies and related
pedagog ies (UCISA, 2010). More recently, our current government has made hints that it will be
prioritising that agenda further. In a recent speech [01/12/11], Michael Gove clearly s ignalled the
importance of fostering digitally literacies within the profession (DfE, 2011) and promised policy
for the new year.
One of the important responsibilities of research in this area is to interrogate the technological
determinacy of solutions being advocated within educational policy, to ensure that we are not
building practice upon potentially naive conceptions of change. The disco urse surrounding
technology, like that of neuroscience, is highly compelling, and the academy has a particular
responsibility to play an initially sceptical role (Pickering & Howard-Jones, 2007) .
In summary, this context serves as a strong rationale for a project such as this. In addition to
contributing to a developed understanding of design for blended learning in higher education, itwill provide my students with an enhanced experience and perhaps most importantly : develop my
capacity as a digitally literate professional.
Context: online learning spaces
For this project I will be focussing on a particular aspect of blended learning: the design and
evaluation of an online learning space (MATHAGOGY). Virtual spaces differ from physical
spaces in several ways, most notably for this project: in theirasynchrony. This refers to a certain
degree of freedom from the constraints of time and affording learners access and interaction
whenever and from wherever they wish.
There are a wide range of online spaces that are currently used for learning. To help map this
territor I will describe these s aces accordin to the model o learnin the redominantl
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/WkwtShttp://goo.gl/l5aKN -
7/28/2019 Capturing Learning Paper
10/48
espouse (acquisition, participation, or a mix), and the extent to which they are managed by
institutions. Fig. 4 below off ers a comprehensive visual representation of this typology.
Fig. 4Map of online learning spaces: click for acronym definitions (Pea-Lpez, 2010)
The first major category of space, commonly known as a VLE (Virtual Learning Environment), is
usuallyfully managed by institutions and predominately based on an acquisition model of learning,
offering students acce ss to c ourse materials (eg. powe rpoint presentations, readers, hyperlinks)
although these is evidence that this focus is beginning to broaden (UCISA, 2010).
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/l5aKNhttp://goo.gl/qexEqhttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/09/ple-shere-e1324904035405.jpg -
7/28/2019 Capturing Learning Paper
11/48
One major criticisms directed at VLEs is that they tend to sustain existing models and perceptions
of learning rather than engaging with the possibilities for learning offered by the affordances of
new technologies (Laurillard, 2007). The widespread adoption of VLEs also appears to have
limited the discourse surrounding blended learning. In 2009, Ofsted reported that the best learning
environments gave learners the opportunity to reinforce aspects of their work as well as the
chance to catch up on missed material. This narrow pedagogical view of the effective use of
VLEs in higher education was characterised by a stark absence of referencesto participatory tools and models of learning.
The second kind of space, a PLE (Personal Learning Environment), is used more frequently by
learners in their everyday lives but is rarely referred to as a learning space. It is not formally
managed by institutions and draws on mixed modes of learning. A PLE is typically the set of tools
and spaces a learner uses to support their learning. The combination of tools used is based on
perso nal needs and prefe rences, and may include social networking sites , discussion forums,
bookmarking services , video curation, wikis, blogs, and so on. The possibilities are vast see Fig. 5 for an example PLE too lset.
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/jPH1K -
7/28/2019 Capturing Learning Paper
12/48
Fig. 5Example PLE toolset(Villaret al, 2010)
One of the major problems associated with PLEs in education is that their effectiveness is
dependent on the autonomy and digital literacy of the learner, and that as a consequence: they
favour the academically prepared (Wiley, 2011). Another issue is that of tool fatigue humans can
only manage so many tools at one time. How can we decide what to choose and invest in amongst
the growing variety of alternatives?
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/FVO04http://goo.gl/53mLXhttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2011/12/PLE-e1324987999473.jpg -
7/28/2019 Capturing Learning Paper
13/48
This leads us to the third kind of space, not yet clearly defined, but emerging in higher education.
This is a hybrid space a fusion between the VLE and PLE (Pea-Lpez, 2010) ,supported
rather than managed by institutions, and based on mixed models of learning as required. This may
be viewed as a way of scaffo lding learners in the development of their own PLE, and as a
consequence: their autonomy and self-directness as learners.
The big challenge raised by this third type of space is the immense complexity of its
implementation. Consider Facebook. I have been told that many of my students use Facebook to
complement their formal learning experiences. What would be the impact of an institutionalisation
of that space: on identity, on ownership, on agency, on autonomy? What would be the ethical
implications for higher education institutions of the corporate interface: of practicing in the public
domain; of the status of our data? And if instituions tried to set up an alternative social
networking service, within the walled garden of the VLE, would people ever really engage with
it?
It is precisely the complex nature of this third space that makes it the most appealing for me. It
raises emerging issues worthy of attention. Academics have access to a growing set of tools for
enhancing learning, and a responsibility to challenge dominant acquisition models of learning
(Laurillard, 2007). Furthermore, the design of this third space is accessible to individual academic
staff. Web 2.o is providing an abundance of free tools for every occasion. I have more choice than
ever in personalising learning experiences for my students. I choose the tools, set up the spaces,
facilitate the learning. Digital design becomes just another wing in my pedagogical repertoire, the
broadness of which impacts directly on the diversity of learning expe riences available to my
learners.
Approach: critical realist hacking
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/qexEq -
7/28/2019 Capturing Learning Paper
14/48
As part of my EdD journey I have been working hard to understand both my assumptions about the
world (my ontology, epistemology and axiology), and how I have arrived at them. I refer to these
areas of my knowledge as myphilosophical position because in my experience they are fairly
cognitively structural. Like religious beliefs they are ideas upon which my understanding of the
world rests, which colour how I see the world. They are based on intuition as much as
reasoning, although I realise it is not quite this straightforward as I digest a body of literature that
reso nantes with my paradigmatic thinking, that process c hanges how I think, although to what extentthese c hanges are structural is beyond my grasp at the moment.
One of the things I have come to realise is that my philosophical position is heavily influenced by
the professional identities I have performed in the past, namely those ofengineer and teacher.
Bec ause I have spent time living and working within both natural and social sc ience paradigms, I
have found it a real struggle to reconcile the tensions between their worldviews, and identify a
clear philosophical standpoint.
Two major breakthroughs have really helped me in this pursuit: the discovery of critical realism
and hacker meta-theories. As I result I suggest that this project (and my research in general) is
bes t described as being underpinned by a realist ontolog y, a constructivist epistemology and a
hacker axiology. So what does this mean in practice? The post-positive ontology of critical
realism is based on the following three assumptions:
1. There is a single reality, which extends beyond the self, but which is only approximately and
no t wholly knowable (Krauss, 2005). Critical realism distinguishes clearly between ourknowledge about the world, and the world that is the object of that knowledge. This is in
contrast to the constructivist assumption that a different reality exists within each persons
conscious.
2. Reality is an open system with emergent properties. This means that it is not reducible to its
constituent parts. This differs form positivist positions which view the world as a deterministic
machine, or constructivist positions which espouse an ungrounded shifting sea of cultural
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
15/48
meaning (Burgoyne, 2010).
3. The emergent nature o f reality means that that general laws which explain the s oc ial world
are unlikely to exist. Instead, critical realists aim to uncovercausal mechanisms that connect
structure and agency within our world. Some of these mechanisms are only available to the
researcher through the events that they cause at an observable level (Crawford & Wright, no
date). For me, structure refers to the enduring forms and outcomes of social practice, and
agency refers to a human capacity to choose and actupon those choices. Structure can enableor constrain agency (Scott, 2005).
So how do we come to know about the causal mechanisms that explain our reality? Critical realists,
like constructivists, believe that knowledge is constructed within our neural systems. As such,
people of fer diffe rentperspectives on reality, and so a large aspect of social inquiry concerns
itself with investigating qualitatively the meanings that people make of the world (Carter & New,
2004).
Although critical realists prioritise peoples descriptions of their experiences, they acknowledge
that these perceptions are historical, value-laden, situated (Carter & New, 2004), and cognitively
biased (Wikipedia, 2011). The plastic nature of our knowledge makes the an awareness of the
positionality of the researcher crucial (Krauss, 2005). It also means that we can never achieve
complete accuracy in our description and explanation of reality. Furthermore, because our
descriptions of the world have the capacity to change the world (and in doing so make themselves
redundant) we will always be at least one step behind the evolving, emergent and looping nature of
reality (Scott, 2005). Despite the risk posed to prediction by the fallibility of thispos ition, practical adequacy canstill be achieved when the resultant knowledge is judged useful
and effective in a contemporary context (Sayer, 1992 in Crawford & Wright, no date).
The values that I aim to espouse resonate with those practiced by the hacker community. When I
talk about hackers I am not referring to the narrowly defined stereotype promoted by the media: the
closet geek who endangers national security by cracking national security networks. This is a
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/S8b7X -
7/28/2019 Capturing Learning Paper
16/48
cracker. A hacker is someone who explores systemic knowledge structures and learns about
them by making. They teach themselves and each another because they are at the bleeding edge
of knowledge about that system (Suiter, 2010). Hackers value playing, making, innovating and
sharing as much as knowing. Their art is bricolage re-imagined for a time when computers have
replaced magic (Suiter, 2010) and their products are tangible, functional examples of knowledge in
practice (Eraut, 1994).
Approach: design-based research
One of the challenges of this project (and my EdD journey so far) has been to find a methodology
that aligns with my philosophical position. In particular, I needed something that embraces the
hacker epistemology of making as knowing, and that can lend itself to the critical realist
investigation of agency and structure.
Design-based research is my best fit find so far. It is an approach that arose from the ashes of
failed design experiments in education during the last century (Van den Akker, 2006). It was
recognised that such a positivist approach was unlikely to yield much of value, but that
investigating design for learning in natural settings still had a lot of potential for the improvement
of theory and practice.
The approach gained momentum during 2003-2006, witnessing several dedicated journal special
issues (Educational P sychologist and Educational Researcher). Things then went a bit quiet in the
last 5 years, but is a new book being released in March (McKenney & Reeves, 2012). I am
looking forward to this in the hope that it will clarify the direction of design-based research.
Currently, proponents espouse a diverse range of positions within post-positivism, and the
approach lacks a clear sense of paradigmatic location. I suggest that it espouses a realist ontology
as it claims and generates transferrable design principles, although there is a lack of consensus
as to the context-dependent nature of these.
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/i6mX0http://goo.gl/zZT7Ihttp://goo.gl/n1401http://goo.gl/n1401 -
7/28/2019 Capturing Learning Paper
17/48
The epistemological assumptions underpinning current design-based research are not so
clear. However, for me (coming from a critical realist perspective) the approach is less about
isolating variables and generating context-free generalisations, and more of a kind
of interventionist ethnography (Dede, 2004). It is experimental, but not an experiment (Kelly,
2006).
I see the aim of design-based research as to improve learner experiences by deve loping theory,
artefacts and practices, through looking at the process of learning and the means that enable and
constrain it in naturalistic settings. It may be further characterised as:
Improvement oriented utilising pragmatic approaches aimed at making improvements rather
than being problem-oriented
Iterative following a repeating cycle of design, evaluation and theorisation
Process oriented seeking to understand what happens during the activity in question avoiding a black box model of input-output analysis
Learning foc ussed looking at how learning happens in the context of the intervention, and how
the intervention influences learning
Utility oriented evaluates the design according to its value for users in real contexts - ie.
impact on practice
Theory oriented design should be grounded in theory and evaluations build on, and adds new
insights to theory (Van den Akker e t al, 2006)
Pedagogically situated has a sharp focus on the underpinning pedagogies of design, andfosters the explication of learning and teaching value positions
Design situated recognises and values the (currently underrepresented) centrality of design
practices in education
In an over-simplification of the process: design-based researchers collaborate with educators to
design a theoretically informed artefact (resource) or process (activity), trial it in a natural setting,
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
18/48
evaluate it, theorise the f indings and then start all over again from a more informed position.
The principle reason this may be a wise choice of methodology in our current educational climate
is down to the potential impact factor of design-based research on policy and practice. I do not
have enough authority to evaluate the influence of educational research on policy, but the
literature sugges ts that it has been limited. I suspect this is in some part due to the different goals
of researchers and policy makers, but Dede (2004) suggest that it is also because educationalresearch is sometimes too theoretically focussed, and conducted with policy and practice only
loosely in mind.
Design-based research has been proffered as a vehicle to foster wider improvement, not just as
an end in itself. The visibility of its outcomes (well-designed artefacts in particular) together with
a strong sense of empirical inquiry could well make an appealing case for political
attention. However, at least equally important is the direct influence of design-based research on
practice . It creates a situation where educational des ign becomes infused with, and stee red bytheory. When contrasted with examples of poorly-grounded educational designs orretrospective
research, it seems an obviously sensible approach.
From a realist perspec tive, design-based res earch can generate plausible causal accounts because
of its focus on learning processes (DBcollective, 2003), and its location in natural settings. This
means that resulting designs are a good fit for the demands of practice, and theorisation has the
potential to provide more robust explanations (Sandoval & Bell, 2004). Fo r this project I will be
using critical realism and a hacker ethos to steer my design-based research methodology. Inparticular, I will be looking at the relationship betwe en the structure of the des ign and agency of
the learners. The hacker ethos suggests that intuition and imagination hold equal status to theory in
design, and so I will be using my own craft knowledge as much as theory to build the online
space.
Whilst one of the main objectives of design-based research is the development of theory, Amiel
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
19/48
& Reeves (2008) argue that this might only occur after long-term engagement with multiple
iterations. As a result I will need to be sensitive to the credibility of any claims I choose to make.
The investment (both in human capital and over time) required to make design-based inquiry
worthwhile could be seen as one of its greatest weaknesses. However, I suggest that the cost of
not infusing design with theory may be greater still. To balance this equation Edelson (2006)
suggests that any intervention must be sufficiently innovative to be worth the investment risk. This
immediately creates an ethical tension: is it permissible to experiment with learners experiences inorder to improve them? The whole of the education system is predicated on incremental
improvement through innovation and evaluation. If there is an issue here, it is endemic and beyond
the immediate scope of this report.
The final challenge faced by design-based research is the demand put on researchers when they
(as in my case) are also acting as a designer. Success requires the bridging of multiple domains
of expertise: research, design, pedagogy, and subject specialisms where appropriate. This mixed
positionality becomes increasingly complex as the researcher-designer endeavours both to makethe design work well at the same time as trying to evaluate it (Sandoval & Bell, 2004). To mitigate
this risk, I am aiming to be reflexive throughout the project.
Approach: integrated methods
This study is guided by the following two questions:
1. How do learners learn in the online space I have designed?
2. How does the design of the online space influence their learning?
To help me do this I will be employing an integrated mixed methods framework (Castro et al,
2010). Mixed methods is a common approach for those conducting research using critical realism,
and strives to generate representative descriptions and explanations of reality, balancing demands
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
20/48
for accuracy, with the pursuit of generalizability. Integrated mixed methods is a particular
approach to using mixed methods that supports asimultaneous rather than sequential analysis of the
data generated from the different methods. This approach allows for a concurrent, integrated and
unified process of data analysis which places an equal status on both qualitative and quantitative
methods (Castro et al, 2010). However, because the epistemology underpinning critical realism is
constructivist, the methods selected will lean more towards the qualitative end of the spectrum of
choice.
Mixed methods have been gaining popularity over the last few years (Wheeldon, 2010). I am
aware that choosing to use multiple approaches to developing knowledge means that, for a finitely
resourced project such as this, I will be spreading my time and attention more thinly than
otherwise, and will likely use several methods more superficially than I would if I was using only
one. However, the realist demand for generalisation means that this is a sacrifice that has to be
made. Mixed methods will allow me to explore the consistency of findings obtained though
different instruments, and to clarify and build upon the results of one method with another(Wheeldon, 2010).
The group I have designed the online space for is made up of a mix of postgraduate and
undergraduate students in their final year of initial teacher education. There are about 60 learners of
a fairly demographically diverse nature, drawn together within the aim of becoming secondary
school teachers of mathematics. At the start of this year the group attend university based sessions
for the first nine weeks before they go into school on a full-time placement. It is during these first
couple of months that they will be encouraged to engage with the online learning space I havedesigned (MATHAGOGY), at the end of which time I will generate data for analysis, evaluation,
and possible theorisation.
For this project I have chosen to use a series of semi-structured focus groups together with a
qualitative survey. I have chosen to use focus groups rather than individual interviews because of
the inductive nature of this inquiry I anticipate that focus groups will encourage participants to
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
21/48
respond to and build upon each others contributions, in a similar way that elicitation techniques
are used in individual interviews (Cohen et al, 2007). I hope to get about six participants (10% of
the population) involved in the focus groups a sample size that will lead to a reasonable
likelihood of reaching saturation in my data analysis (Castro et al, 2010).
I recognise that there are some important disadvantages to using focus group methods. For this
project in particular, it means I will miss out o n the oppo rtunity to talk to participants alone and inconfidence. In a peer group situation they may be less likely to share information that could
damage their own, or other participants reputations or feelings. However, on balance I am
convinced that for the purposes of this study, the benefits of grouping participants outweigh the
disadvantages, partly because they will have an opportunity to contribute in confidence as
individuals in the project survey.
I have chosen to employ a compact, semi-structured, qualitative survey (Cohen et al, 2007) to
complement the data generated from the focus groups. The survey will be sent electronically to allparticipants (60 or so) soon after the focus groups are conducted. This means that anyone who
wants to say something more has the opportunity to do so without delay. The survey will pose two
fairly open questions and offer a large amount of space to respond in text. The phraseology is
designed to be open and familiar:
1. Have you learnt anything from MATHAGOGY?
2. How could we improve MATHAGOGY for the future?
One of the major limitations of this study is its lack of consideration of natural data during the
analysis stages. All participant comments are openly accessible and recorded on the learning space
it would be technically straightforward to retrieve this data, creating an additional perspective on
reality. However, I have chosen not to do this for two reasons. Firstly, this is only a pilot project
and has finite scope and resource. Secondly, because at the beginning of the project, students
were not informed that their comments would be analysed. The ethical issues surrounding the use
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
22/48
of open data are emerging and complex, but the analysis of natural data is something I do wish to
explore in further detail in preparation for the next stage of my EdD.
All participants were made aware from the beginning that the design and evaluation of
MATHAGOGY was the focus of a research project for my (their lead tutors) EdD. In addition,
those who attended the focus groups were made aware that I was recording the discussion, and the
primary data was dele ted so on after it has been processed.
What actually happened
Everyone in the group was invited to join me for focus group sessions on the ninth friday
afternoon following lectures. Nine students chose to attend (a 15% self-selecting sample) and I
saw them in groups of threes. The focus group was fairly loosely structured: I posed an opening
question: tell me about your MATHAGOGY experience, let them speak, and probed further
where I felt there was something interesting behind what they were saying, or if I neededclarification. The focus groups came to a natural conclusion each time between 30-40 minutes. I
made memos of my thinking during the sessions, and audio-recorded the discussions on my
smartphone fo r subsequent analysis.
Clearly these students are the ones who are more likely to have engaged with the online space and
so are not an unbiased sample of the population. However, for the focus groups this was not
significantly problematic ,as I was striving to generate data about those learning experiences
that did occur rather than how frequent they were. The survey was designed to explore therepresentativeness of the student experience. It had an approximately 75% response rate (n = 46)
which was enough to give me an insight into how the experiences of the focus group participants
extrapolated across the population.
A roach: critical realist analysisPDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
23/48
In order to maintain a coherent epistemological approach I will be using a critical realist analytic
framework to guide the analysis of my data. The end goal of this is to identify causal
mechanisms which explain how structure and agency in the context of this project are related. To
do this I will be using a modified version of Danermarket al.s (1997) six stage model of
explanatory realist research.
Stage 1: Description In the first phase of analysis I will be aiming to generate a rich description
of both the product (an analysis of the webspace design) and process (how people perceived they
learned through the webspace), using the variety of data available including: students responses
during the focus groups and survey; my field notes; and analytics available via the learning space
software. It is important to note that I am not trying to describe how people have actually learned
via MATHAGOGY at this stage (as a critical realist) I cannot claim anything beyo nd an
interpretation of theirperceptions.
In my analysis of the learning process I will use words and phrases used by the participants as
much as possible, to create authentic and rich descriptions in preparation for the next phase of
analysis.
Stage 2: Decomposition During this phase I will aim to dissolve the complex and composite into
a variety of dimensions. This is not the same as practicing reductionism emergent processes
are greater than the sum of their parts, and so the concept of dimensions rather than components is
more appropriate here.
I am aware that I will have a significant influence over how the data will be divided up, and where
the boundary of interest lies (Burgoyne, 2010). This is okay it is not possible to look the data
from every possible angle, particularly in an open social system such as education, where there
are limited clear boundaries. This is where my research questions will play an important role, in
steering the analytical process:
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
24/48
1. How do learners learn in the online space I have designed?
2. How does the design of the online space influence their learning?
Stages 3&4: Abduction & Retroduction These two stages will be conducted concurrently as
the value here lies as much in theirinterplay, as discretely considered processes. Abduction refers
to the interpretation and redescription of the dimensions highlighted in the previous phase. It is theconstruction following the deconstruction phase, but now with the aim of building an abstracted yet
conceptually coherent model of reality.
As this phase progresses I will need to explore and exploit relevant conceptual and theoretical
frameworks in the development of a robust explanatory model. However, due to the inductive
nature of the project, it is not appropriate to pre-identify any of these, as it might risk
(further) limiting the scope of my imagination and reasoning significant aspects of researcher
thinking during the process (Hume in Stevens, 2009).
Retroduction refers to the process of highlighting various dimensions and asking myself questions
such as:
How are they related/unrelated?
What properties underpin them?
What else is going on here?
What causal mechanisms are at play?
Together, abduction and retroduction can result in the uncovering of underlying mechanisms (how
learning is happening/not happening in and because of MATHAGOGY) and a potential theorisation
of the process (Livock, no date). One of the main challenges here is to distinguish between cause
and correlation. Whilst we can observe correlation we cannot observe cause we infer cause.
Another is to sensitise myself to how the data and my values are intertwined (Carlsson, no date).
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
25/48
Reflexivity plays an important role in addressing my positionality I will be keeping one eye on
the analysis and one eye o n myself throughout.
A keen awareness of the fragile nature of any knowledge claim lies at the heart of the critical
realist paradigm. It is important to recognise that this is my first real attempt at trying to analyse data
in any depth. As a result, I am sceptical as to the credibility of any theorisation that I may claim to
achieve. Nonetheless, I am going to have a good crack at theory building as I hope to learn (atleast) as much from the engaging with theprocess as I do from the project outcomes.
Stage 5&6: Redescription & Recontextualisation In these final stages, the scene is set to look
at any theory that I develop through the eyes of existing theoretical frameworks, as well as
interrogating their explanatory power in the wider context of online learning spaces in higher
education.
Analysis: describing the design
This section of the report details the design of the online learning space (MATHAGOGY), and
briefly explores the mode l(s) of learning that it espouses, or rather thatI have espoused as the
architect. It is not a section that appears in a typical educational research report, but is an essential
requirement of the design-based methodology employed in this project.
The goal of critical realist research is to uncover the underlying causal mechanisms that link
structure and agency. Within this project, structure refers to the design of the learning space, my
role as a tutor and the culture that emerges within the space. Without an explication of these first
two elements there would be no structure to relate to in the structure-agency analysis. The third
element is a little different because it is emergent rather than designed, and so falls within the
analysis of participant experience. There is of course a much greater range of structures that
enable or constrain the agency of participants, but I have drawn a boundary around the
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
26/48
aforementioned aspects to provide the inquiry with a sufficiently sharp focus.
An obvious critique of this research design is that it deviates from suggested design-based
research methodolog y by notgrounding the learning space design explicitly in theory. As this was
the my first research iteration I decided to generate a design grounded foremostly in my own tacit
expertise (Eraut, 1994), and to use this as an opportunity to explicate my pedagogy (Swan, 2008).
This may limit the potential of theorisation during analysis, but the uncovering of my values holds agreater long term return-on-investment value in a pilot project such as this. As a result,
successive iterations will be grounded in both theory, and a clear understanding of my own
assumptions and practice as a designer.
There are four major components of the MATHAGOGY architecture: environment design; content
design; interaction design; and role of the tutor:
Environment design The space itself is a fairly formal, uncluttered, and focussed on a singlecore activity: the presentation of relevant artefacts with the opportunity to comment on them. The
use of a logo has been employed to give the space a more aesthetic appeal, arguably of equal
importance to functional effectiveness (Burkhardt, 2006).
Content design Content was curated based on my subject expertise. Artefacts (articles, videos,
websites etc.) were selected for both their mediaformat (a variety was deemed desirable) and
potential to challenge participant pe rceptions of what it means to be a maths teacher in the digital
age. The abundance of information available on the internet in our times means that finding andidentifying highly relevant artefacts requires a certain degree of digital literacy expert curation
is a digitally literate pedagogy underpinning this aspect of the design (Weller, 2011). Content was
deliberately presented neutrally, forcing participants to position themselves when commenting on
the artefac ts.
Interaction design One of the major design features of MATHAGOGY was the provision of a
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
27/48
space to comment on artefacts. This demonstrates the value I place on participation models in
addition to acquisition models of learning, where learning occurs through intellectual activity rather
than just via the absorption of information (Mayes & de Freitas, 2007). Commenting on and
collaboratively discussing are processes rooted in Vygotskyan (1978) and Bahktinian (1986)
notions of social constructivism and conceptual development.
Role of the tutor (me) My role in MATHAGOGY is characterised as: designer of the learningspace; curator of relevant content; and lurker in the interaction space. The absence of my voice
in the comments is a deliberate decision, not based on time constraints, but rather on the premise:
as soon as you have an expert in the room, people defer to them. Instead, participants were left to
figure out their positions together, learning from in each other, congregated by a relevant social
artefact (Littlejohn, 2011). This resonates with learning processes described by communities of
practice theory (Wenger, 1998).
Overall, the design is based on a mixture of acquisition and participation models of learning,valuing both the assimilation of a body of knowledge as well as the development of traditional
and digital literacies (Beetham & Sharpe, 2007), where participants can practice critical reflection
and perform their values (Adhikari, 2011) in an authentic professional setting (Eraut, 1994).
Analysis: describing learning
This section offers an overview of how I handled the data (generated by the focus groups, survey
and field notes), with an insight into the description phase of the analysis.
The first part of the process involved semi-transcribing the focus group recordings. I did this by
listening several times to the audio recordings, and each time capturing what I felt were significant
soundbites on Idea Sketch (a concept mapping tablet app). These were then supplemented by
soundbites harvested from the survey, and from my field notes, to produce the soundbite
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/vhd9G -
7/28/2019 Capturing Learning Paper
28/48
overview illustrated in Fig.6 below:
Fig.6Screenshot of soundbite overview (click for larger image)
The reason I chose to only semi-transcribe the captured data is because I wanted to prioritise
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/photo-2-e1327047771392.png -
7/28/2019 Capturing Learning Paper
29/48
the connection of concepts from early on in the process. Using concept mapping rather than
verbatim transcription meant I could amend and move around the soundbites with each listen. At the
end of this process I had a loosely in vivo themed overview of the content which I was able to
export to excel and tag with (what I felt were) appropriate dimensions (see Fig.7).
Fig.7Screenshot of the attributing dimensions process (click for full dataset)
Following this I was able to filter the dimensions and get a sense of recurring and/orsignificant
themes. A crude quantitative analysis of the attributed dimensions is illustrated in Fig.8 below
(Wordle uses word size to represent occurrence frequency).
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/Screen-shot-2012-01-20-at-08.42.38.png -
7/28/2019 Capturing Learning Paper
30/48
Fig.8A Wordle illustrating the relative frequency of attributed dimensions
This crude quantitative analysis was used in conjunction with the earlier, more qualitative analysis
(where I filtered each dimension in excel, and then was able to further connect the soundbites
which were attributed the same dimension) to create a sense-making artefact (Siemens, 2005)
which I could then take my analytical knife to. This process of making as understanding is an
important part of the hacker epistemology which underpins and runs through this project. The
resulting artefact is presented below:
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/M8qRhttp://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/Screen-shot-2012-01-22-at-10.37.58-e1327655557735.png -
7/28/2019 Capturing Learning Paper
31/48
Your tutor gives you a link
He says this is our forum and tells you to get involved
You want to impress him
So you have a look
It is a simple website with some writing
And videos and articles and links
All of its interesting
And it doesnt look like too much
So you read the first post
Hes asked you to comment
You go to write something
But youre not sure what you think
It takes you a while
You figure out where you stand
Youre not quite comfortable
But you give it a shot
Now youre writings out there
You feel a bit exposed
Everyone can read what youve written
What will they think?
Next week theres another post
You read other peoples comments
They change how you see things
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
32/48
You reluctantly continue
The ideas each week are great
Where does he get them?
Youd prefer just to read them
Commenting requires so much more thought
You start to feel more confident
Its like practicing something
MATHAGOGY feels like a safe space
But is it really?
Your workload is mounting
Youve no time to read or comment
But at least itll be there
If you want to look back
At this point I felt that I had described the perceived learning experiences of the participants in
enough range and detail to be able to begin deconstruction. The one thing I have felt uneasy with is
the centrality of my intuition and imagination during this process. My past identity as an engineer
screams use your reason man at me, but I take comfort in deferring to Hume (in Stevens, 2007)
who suggests that imagination plays a role of equal importance to reasoning, when conducting
social research.
Analysis: explaining learning
In this section I present an overview of the the abduction-retroduction process and its outcomes: a
set of causal mechanisms, framed as design principle hypotheses. The emphasis on design is an
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
33/48
important part of the design-based research methodology employed by this project the
hypothesis of contextually-sensitive transferrable principles allows for the theory emerging from
this analysis to be implemented, evaluated and refined in subsequent iterations (Van den Akker,
2006).
As discussedpreviously, the abduction and retroduction processes are conducted simultaneously,
in an attempt to identify connections between structural and agentic aspects of the perceivedlearning experience, with the aim of distilling a conceptual framework from which causal
mechanisms may be inferred. A snapshot of the retroduction process is captured here. During this
phase I pos ed and tackled questions such as:
What is happening here?
How are the dimensions related/unrelated?
What properties underpin them?
What else is going on here?What causal mechanisms are at play?
During this process certain relationships between dimensions began to emerge as significant. The
majority did not. The diagram below (Fig.9) represents the main dimensions identified and how
they are connected.
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://goo.gl/m9lShhttp://goo.gl/6kbEY -
7/28/2019 Capturing Learning Paper
34/48
Fig.9Snapshot of the abduction process (click for larger image)
These connections were subject to substantial interrogation as to the nature of their relationship:
incidental, correlative or causal. Those that appeared to be causal were reframed as design
principles hypotheses . I have made hypothetical rather than theoretical claims at this stage because
neither my data analysis framework nor personal analytical aptitude warrant claims of any causal
certainty.
I used two frameworks to help guide me during this latter phase of analysis. The first was the
ro ect research uestions:
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/01/photo-3-e1327048540813.png -
7/28/2019 Capturing Learning Paper
35/48
1. How do learners learn in the online space I have designed?
2. How does the design of the online space influence their learning?
And the second was a loose typological framework developed by Illeris (2007) in his
comprehensive synthesis oflearning theory literature. As a result, I organised my design principle
hypotheses by how individual or (more commonly) combinations of structural dimensions enabledor constrained certain types of learning. Below is a list of the design principle hypotheses (DPH),
organised by learning type, supplemented with examples of soundbites harvested from the focus
groups.
Assimilative This refers to learning experiences that do not disrupt current schemas. New concepts
can be addedto existing f rameworks without significant overhaul.
DPH1 The greater the abundance of subject information, the greater the value of curation as apedagog ical activity
Where are you getting these amazing things?
I was able to acces high quality content that would have taken me years to find
DPH2 The greater the perceived subject expertise of the curator, the greater the perceived
value of the content, even if that curator is just repurposingexisting content (Siemens, 2005)
You take it seriously
More educational value than other blogs
Accommodative This refers to learning experiences that require making room for, where
existing framework have to be modified so that new concepts can be integrated into personal
knowledge networks (Illeris, 2007).
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
36/48
DPH3 The more the online learning space is perceived as being safe yet public, the more
likely participants are to perfo rm their professional identity
An opportunity to try ourselves out as maths teachers
Room to explore your thinking
You know other people are going to read it
DPH4 The greater the opportunity for participant voice, the greater the potential for identity
development
I spent an hour just thinking what to write
Where do I see myself fitting in with the ideas
It represents you
Transformative This refers to learning experiences that result in a capacity change. Existing
concepts and frameworks are reframed and reinterpreted, destabilising personal knowledge
networks, and allowing for future learning to happen in a different way (Mezirow, 2009).
DPH5 The greater the opportunity to be reflexive about their learning (including identity
development), the greater the potential for sustained capacity change (as learners)
Makes you a more critical personIve changed how I comment now when I read others views it make me think
differently eg. did I read that right?
Disclaimer As I write these I am acutely aware of the challenge presented by the original
contribution to knowledge criteria of the thesis phase of the EdD. What I have generated here are
hypotheses: suggested relations that require rigorous exploration before they could become
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
37/48
claims that could be generalised, even in a context-sensitive manner. There are at least three
significant challenges which I will need to get to grips with andfigure out if I am to fulfil this
criteria in the future.
The first challenge is to understand how to unpick the the influence of the design, when compared
with the influence of the implementation (Ravenscrof t, 200 7) of online learning spaces . In the case
in this study, the design is highly visible, but the implementation is much less so. If MATHAGOGY
was implemented on a similar course in a similar context, but different tutors and learners, how
would the experience differ? This leads me into my second challenge:
The enormous complexity of the general process of learning, and as a result: the daunting
challenge of identifying causal mechanisms that influence it. This complexity is confounded by the
limitless social, cultural and political forces that contextualise and influence education (Selwyn,
2011). And finally, the challenge of unpicking learning andperceived learning. With the methods
applied in this project what position am I in to make claims about actual learning? What value do
perceptions of learning hold? And how can my expertise as a choreographer o f learning help me
to evaluate learning in a meaningful way?
Interestingly, the last two of these appear to be largely the result of a conflict between the realist
ontology and constructivist epistemology of the paradigm that I am operating within these issues
are disc ussed in more detail in the Limitations chapter of this report.
Analysis: theorising learning
This section attempts to explore my analysis and DPHs (Design Principle Hypotheses) through
the lens of transformative learning theory (Mezirow, 2009), with the ultimate goal of building on,
or repurposing some o f that body o f understanding. At this s tage it is important to note that:
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.capturinglearning.com/?p=204 -
7/28/2019 Capturing Learning Paper
38/48
Due to the pilot nature of this s tudy any knowledge claims will be tentative and hypothetical.
I have yet to fully articulate what theory looks like within a critical realist paradigm. Is a set of
design principles a sufficient theoretical contribution (as would be accepted in the world of
natural science), or in a social science context does theory have to be somethinggreater?
As discussed previously, transformative learning refers to cognitive capacity change within the
learner, or what Mezirow (2009) would describe as a modification of our frame of reference(aka. mindsets, habits of mind, meaning perspectives). This can be contrasted with informative
learning, which happens more often as a result of assimilative and accommodative processes (as
described in the previous chapter) (Kegan, 2009). Epistemologically, these alternatives may be
thought of as the difference between what we know and how we know. Mezirow (2007)
argues that the latter is significant for learners, because new ways of knowing are more likely to
generate beliefs and opinions that will prove more true or justified to guide action (p.92).
This project may be able to contribute to transformative learning theory in two aspects, both ofwhich are under-represented in the literature I have been exploring. The first is in connecting
agency and learning within Mezirows (2009) framework, and the second in clarifying the
relationship and distinction between informative learning and transformative learning, with
acquisition and participation models of learning.
1. Connecting agency and learning within transformative learning
In my data analysis, the notion of identity performance emerged as a significant process infostering perceived learning via MATHAGOGY. I have positioned it
under accommodative learning processes (DPH3&4), but with the potential to become
transformative if performed reflexively (DPH5). What is significant here is that this
reflexiveopportunity is directly linked to the capacity of the learner to re-construct and re-
perfo rm their identity (o r self): a process that resonates closely with Levines (2005) description
of agency. I suggest that this reframing ofreflexive identity re-performance as agency may be a
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
39/48
useful theoretical contribution when considered alongside the growing body of structure-agency
theory, as it has the potential to help educators make more informed decisions when designing for
transformative learning.
For example, if I want to build a online learning space to foster transformative learning I can more
readily attend to the structural factors that may enable or constrain the opportunity for identity re-
perfo rmance, with a clearer vision of how this might impact on my learners abilities to act in
similar (and possibly professional) contexts. Furthermore, I would be able to utilise theoretical
frameworks such as structuration (Giddens, 1984) or morphogensis (Archer, 1988) to help me
understand how my design can lead to transformative experiences, and how these changes in
agency might then modify how the learning space is appropriated by the learners.
2. Connecting information, transformation, acquisition and participation
When I first began to explore the relationship between these concepts, I was unsure whether
information and acquisition were interchangeable terms. However, after consideration of these
concepts in the context of my DPHs I suggest that although they are similar in the process that they
describe, the use of acquisition is more helpful in a design for learning context as it considers
the activity of the learner rather than any epistemological outcome. This is significant because it
emphasises the affordances of particular pedagog ical approaches.
The relationship between transformation and participation is similar (although arguably much more
complex). Looking back at my DPHs in the previous chapter I can see an emerging correlation
between the degree of participation and potential for transformative learning. Again, this
distinction stresses the impact a particular pedagogical approach may afford for learning
outcomes. I suggest rather than seeing these relationships as discrete dichotomies, they are more
usefully conceptualised as being on competing ends of a spectrum, as illustrated in Fig.10 below.
This is clearly manifest when considering, for example: the role of identity performance in this
study.
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
40/48
Fig.10The information, transformation, acquisition and participation spectrum
Where does my learning during this inquiry sit on the above s pectrum?
This is a relevant question to address if I am to enact the participatory model of learning that I
espouse. In short, I would say that I have encountered a wide range of learning experiences, with
their relevant epistemological consequences. What might be more interesting to explore is how my
capacity for agency has developed, and to what extent I may be re-appropriating the facilitative
structures that have catalysed these changes. However, despite being
an important and interesting question, this lies beyond the scope of this study, and shall have to be
left for another time.
Recontextualisation
In this section I will reflect on my analyses, and consider how they fit with ideas discussed in
the earliercontextchapters. I will attempt to address the following question: how do my analyses
help us to understand the role HLEs (Hybrid Learning Environments) such as MATHAGOGY might
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://www.pepsmccrea.com/capturing-learning-paper/wp-content/uploads/2012/02/Screen-shot-2012-02-17-at-14.19.221-e1329488385185.png -
7/28/2019 Capturing Learning Paper
41/48
play in the unfolding of our turbulent educational landscape over the next fe w years?
To begin, it is worth clarifying whether we would wish for HLEs to play any role in higher
education at all. Is the impact of MATHAGOGY as hypothesised in the DPHs (Design Principle
Hypotheses) consistent with the values of the sector? Despite the lack of clear consensus
concerning the purpose of higher education (Schwartz, 2003) I suggest that the facilitation of
learning is one of its less disputed activities.
Based on the DPHs, MATHAGOGY appears to provide opportunities for a wide range of
higher learning activities (Garrison & Kanuka, 2004), including (based on an amended design)
transformative learning which I suggest offers a rich value proposition for students and the wider
stakeholders of higher education (employers, society etc.). Despite their promise, the literature
suggests that HLEs are more likely to enhance education by being integrated with traditional face-
t0-face experiences rather than replacingthem (Haythornwaite & Andrews, 2011).
In addition to enhancing the provision of learning experiences, I propose that HLEs like
MATHAGOGY have the potential to impact on higher education in two other significant yet
argulably less visible ways. Firstly, by enhancing learner digital autonomy. As students practice
learning in a hybid safe yet pubic space over a period of time (see DPH3), they become
increasingly exposed to the potential of PLEs (Personal Learning Environments) as tools that
can facilitate greater self-direction in their learning (Kop, 2011). In effect, well designed HLEs
can sc affo ld the learner in understanding both the how and the why of their own life-long learning.
Secondly, in their ability to disrupt pedagogy and practice, HLEs have the potential to transform
some aspects of higher education (Anderson & Shattuck, 2012). Technology has often been cited
as a catalyst for change (Veletsianos, no date), and in this instance the vehicle is increased
pedagogical awareness (realised through the explication of design decisions) and innovation. This
is in stark contrast to the impact of VLEs (Virtual Learning Environments) on institutions, where
innovations have often sustained existing pedagogies rather than disrupting them (Laurillard, 20 07).
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
42/48
One further potential consequence of the development of HLEs is contingent on who in involved
in their design. If academic staff are part of (or lead) the design initiative for online spaces for
learning, then the institutional capacity for digital literacies is likely to increase (McLoughlin &
Lee, no date). Again, this is in contrast to institutions that operate and rely on centralised VLEs,
where the opposite can happen: academic staff can become deskilled, both in their digital literacy
and aspects of their pedagogical practice, by being offered (and sometimes encouraged) to
utilise off-the-shelf digital tools for facilitating learning.
In summary, HLEs have the potential to be transformative for both the learner, the institution, and
the sector, but only where academic staff take responsibilityforand are participant in the design
process (Ravenscro ft, 2007). Ultimately, this could lead towards new foci for pedagog ical
practice (design specialists, content curators etc.), whist continuing to recognise the role of
teacher as expert (McLoughlin & Lee, no date).
Looking back: limitations
In this section I will explore the major ontological, epistemological, methodological and
axiological limitations and inconsistencies inherent in this study, in terms of research design and/or
implementation as appropriate.
One of the major problems of this research design has arisen from either: an inherent conflictbetween the realist ontolog y and constructivist epistemology within the critical realist meta-
framework, or my interpretation, understanding and implementation of these components. This is
most apparent when considering how learning was evaluated throughout the project, and in
particular the difference between actuallearning andperceivedlearning.
My realist ontology suggests that there is one true way that each of the participants this study
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
43/48
learned via MATHAGOGY. My constructivist epistemology suggests that the best way to find out
is by analysing their representations of their experience. However, this is
problematic because participants may not be aware, or be able to explicate what or how they
learned. This is not to criticise the respondents in any way they are a highly intelligent and
articulate group of people but rather to recognize that all humans have cognitive biases. For
example, in a study of Standford MBA students, 87% of them rated their academic performance
as above the median (Zukerman & John, 2001)! This dissonance between the aims andprocessesof this study means that it could be seen as lacking in coherence, and that even the tentative and
hypothetical nature of my claims may be ove r-ambitious.
Setting this inherent conflict to one side, I believe that my general approach to researching learning
during this study has been inadequate. After working through the process, I now understand the
extent to which research exploring the hugely messy human activity oftechnology mediated
learning, should be required to consider complexity theory (Morrison, 2006). Despite the
abundance of empirical inquiry that has explored the effect of technology on learning, theevidence is overwhelmingly inconsistent (Selwyn, 2011). Notions such as chaos, emergence, and
complex adaptive systems are increasingly being harnessed to help explain social activity,
allowing us to generate more holistic, rigorous, and ecologically sound theories of learning
(Davis & Sumara, 2008).
Another aspect of this study which is a major weakness is the lack of clear position of the role of
theory in design. I do not yet have a clear understanding of what this might look like, and as a
result the outcomes of this study are somewhat confused. Burkhardt (2006) suggests we should be
asking how far current theory is an adequate basis for design. Is a set of design principles for
learning as valid as a socio-cultural theory of learning? And how and who should decide? This is
something I am keen to pick up in the next stage of my EdD: an exploration of the role of theory
in design, design as research, and how these ideas fit with notions of rese arch impact.
I believe the concept of design as research is particularly relevant in our current times.
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
44/48
Haythornthwaite & Andrews (2011) argue that the pace of change, not only of technology, but in
the ecology of education , means that a different kind of research is required: forblended learning
as well as aboutblended earning. They suggest that research which runs hand-in-hand with (rather
than behind) development, which contributes to the design of new products, networks and models
of learning will have significant impact and currency over the coming years. This idea aligns
pleasingly with my hacker axiology: valuing making as knowing, with tools as concepts: to be
understood through use (Collins et al., in Mayes and Freitas 2007).
Looking forward: possibilities
In this section I will talk about the impact and potential of this study: on the field, my local context,
and my practice. Despite the limitations discussed in the previous section, there are several
aspects of this research which may be seen as value-generating. In the main, these are more a
consequence of theprocess engaged in, rather than any resulting findings. This is not necessarilya detriment of the study emerging models of connected learning (Belshaw, 2012) suggest that
in our information abundant context (Weller, 2012) learning experiences are at least as important as
learning outcomes: something which, from my perspective, the EdD lends itself neatly to (certainly
in the first phase).
Firstly, this study has highlighted the potential of online spaces to enable learners in higher
education to practice their voice, identity, and being part of a community. It has: described and
enacted a set of design principles, which have not to my knowledge previously been identifiedand/or described; and highlighted the challenges of figuring out and theorising online learning. In
particular, it has demonstrated how hybird learning environments can offer new avenues for
pedagog ical practice whilst continuing to recognize the role of the teacher as expe rt
(McLoughlin & Lee, no date), for example as: designers for learning, curators of content, and
choreog raphers of reflexivity.
PDFmyURL.com
http://pdfmyurl.com/?otsrc=watermark&otclc=0.01http://pdfmyurl.com/?otsrc=watermark&otclc=0.01 -
7/28/2019 Capturing Learning Paper
45/48
It has already been an interesting case for discussion amongst colleagues in my institution: helping
us to challenge dominant face- to-f ace pedag og ies, and probe the liminal boundaries of
institutional policy. The power of this study lies in the combination of a tangible artefact
(MATHAGOGY), together with an analysis of that artefact. People can see the webspace: its
form, and how it has played out (all the comments are still there). These, along with my analyses,
may allow people to begin to envision how modifications in practice might lead to an impact on
student learning in new ways. It provides a lens for understanding how theory can be translatedinto practice in complex educational settings (DBcollective, 2003).
Above everything else, this study has had an invaluable impact on my own thinking and practice. In
addition to the ideas discussed above, it has enabled me to understand and redefine the balance
and scope of my pedagogical practice. I now view the creation of online spaces for learning as
an option available to me as a designer of learning. And this is particularly significant as the locus
of influence I have over the online environment is much greater than that of the physical. I cannot
move the walls of a classroom, but online I can bend both time and space.
In realising all this I have developed greater agency as an educator. I now have the capacity to
harness particular tools for particular purposes, rather than picking a stock tool off the VLE
(Virtual Learning Environment) shelf. This subtle difference mitigates against design blindness,
making me painfully aware of some of the structural and environmental aspects of the experiences
my learners encounter, over which I have less influence. However, it also mitigates against design
deskilling, as Ipractice making online learning environment