evaluating media literacy as competences: what can we agree on?

EVALUATING MEDIA LITERACY AS COMPETENCES: WHAT CAN WE AGREE ON?

Pierre FastrezGroupe de Recherche en Médiation des Savoirs

Université catholique de Louvain, Belgium

Where do I stand? (a few oratorical precautions)

Pierre Fastrez [GReMS/UCL]

What I intend to bring to the table a social/cognitive/communication scientist’s

point of view a newcomer’s perspective

EuroMeduc Bellaria – Oct. 22nd, 2009

A simple observation…

Pierre Fastrez [GReMS/UCL]EuroMeduc Bellaria – Oct. 22nd, 2009

Media Education, a vivid field Media Literacy evaluation

is still in its infancy still lacks systematic efforts to make it a

coherent endeavour

Assessing ML : what priorities?

Agreement: Regarding theoretical definitions of Media

Literacy Not on definitions and frameworks But rather on how these definitions and

frameworks relate Regarding methodological principles for

assessment

Outline

1. Assessing Media Literacy: what can we agree on?

a. Theoretical issuesb. Methodological issues

2. A few proposals regarding Media Literacy competence assessment

Theoretical issues

Assessing ML requires an agreement on what it is

Defining ML is dependent on Historical and cultural context Media landscape development Policies Epistemological views

Theoretical issues

Goal: identify common ground for Media Literacy definitions Independent of cultural contexts That can be specified and interpreted in these contexts

Media Literacy lies in the individual’s competences not the media environment:

media literacy is not equivalent to media education Outcome vs. process

Evaluating the process is evaluating its efficiency with respect to the outcome

Benchmarking outcomes is a sine qua non condition

Theoretical issues

A fundamental hypothesis: Media Literacy is related to a number of generic skills

E.g. media and communication technologies as mindtools These skills are generic in that they are not related to

specific media (film, print, internet…) specific activity domains

E.g. commercial information, product placement, minor protection, civic participation…

A bi-lateral relationship (virtuous circle) These skills can be developed through specific media use and

specific media education practices The more these skills are developed, the more they facilitate

other media appropriation processes

Theoretical issues

Consequence of this hypothesis: we need to relate specific skills to their “common ground” E.g. “digital” skills are not of a different nature than

“traditional media” skills E.g. Film studies vs. media education

appreciating one’s media cultural heritage relies on the same skills as appreciating other cultures’ contribution to the media environment

E.g. Understanding the semiotic specificities of a given media: Representing time in blogs and twitter (input flow and chronology) Representing time on the silver screen (editing and rhythm)

E.g. Building typologies: reading the news and managing bookmarks

E.g. Perceiving the producer’s intent: watching a documentary and moderating a forum

Methodological principles

Methodological issues regarding ML evaluation Definition of levels Competence and performance Assessment tools validation process

Levels of Media Literacy

There are degrees of complexity in competences

Operationalization: Levels (when it makes sense) Tools for researchers Not meant for unilateral ranking

Which would assume one can weigh the relative importance of skills

Rather, a multidimensional analysis tool

Competence vs. performance

Competences are virtual, and can only be observed when actualized in practical and concrete tasks

Evaluating competence through performance Consequences:

The production vs. reception distinction should be set (for now) at the performance level (vs. as categories of skills)

Performances are connected to different competences E. g. moderating a forum: social and cognitive

competences

From concept to indicators

Concept •Media Literacy

Dimensions •Competences•How can we group them? (technical, social, cognitive)•What levels can we define?

Indicators•Performances: specific tasks•What activity domains?•What media?

From concept to indicators

Concept •Media Literacy

Dimensions •Competences•How can we group them? (technical, social, cognitive)•What levels can we define?

Indicators•Performances: specific tasks•What activity domains?•What media?

2. An attempt at breaking down the Media Literacy concept

Cognitive vs. social(vs. technical) skills

Semio-cognitive vs. socio-cognitive skills

Conceptual distinctions independent of specific activity fields

Representation

Information

Intention

Representation

Information

Intention

2. An attempt at breaking down the Media Literacy concept

Representation

Information

Intention

Semio-cognitive skills

Dimensions (inductive grouping) : Reading (decoding / synthesizing /

evaluating) Writing (coding, creating, producing) Navigating (searching, exploring, managing

uncertainty) Organizing (keeping, sorting, annotating) Using responsibly (knowledge of use

regulation, stakes and principles)

Socio-cognitive skills

Pragmatic socialization: The extent to which an individual poses communicational acts that establish social relationships in more or less extended social spaces.

Informational socialization: The extent to which each individual extends and diversifies their relationships as a receiver, a relay, or a producer of information within social space.

Social levels logic (criteria, not thresholds) From contact and participation, to networking and active

presence, to cooperative networking Diversity (from tribe to society) Creativity and involvement (from follower to autonomous

agent) Decentration (from egocentric to allocentric)

Dimensions: Reception: media selection and exploration Expression: communicative acts, idioms and receivers Pragmatics: relational contexts and interactions Ethics: perceived rules and lines of conduct

Informational socialization

Reception

Expression

Are informational socialization skills only socio-cognitive?

Theoretical questions left open

How are Media Literacy skills correlated? From an analytical to a systemic point of

view: connecting after having distinguished E.g. does critical thinking development

facilitate creativity? Ethics as a socio-cognitive skill and responsible

use as a semio-cognitive skill …

Need to work on the structure of ML as a concept federating a set of competences Not just conceptually, but empirically

Back to methodology: validation

Building relevant ML assessment tools takes time and a lot of research

ML dimensions (skills) can (and must) be translated into multiple indicators related to different media / activity domains

Validation requires Multiple indicators for the same skill Internal consistency evaluation

Back to methodology: validation

Data / indicators based on (roughly in order of preference) Observation of specific tasks

Experiments Participant observation

Analysis of traces Personal environments / personal information spaces Including statistics

Interviews and surveys Self-assessment tools

Triangulation: a requirement for validation

Conclusions and implications

Working on competence definitions and their operationalization is a requirement for ML assessment.

Constructing ML assessment tools are a means for setting objectives for media educators bridging the gap between

asymptotic goals (“be an active citizen and media user”)

and practical know-hows (“know how to frame a close shot”)

THANK YOU FOR YOUR ATTENTION

Questions? Comments?

pierre.fastrez@uclouvain.be

evaluating media literacy as competences: what can we agree on?

Education

abm competences

model for evaluating teacher and trainer competences -...

learning competences as citizneship competences

writing competences

ehes-fy19 title 1 parent survey (responses) ·...

teaching remote sensing competences the implementation...

measuring competences with comet 2013 measuring competences...

competences, bleixen

evaluating competences

key competences

referentiel emplois competences

videoconferencinginteachingcross-cultural competences ·...

competences pecha kucha

competences pcur compét:nccs. ww.anapec -...

reflective team strategic competences, knowledge ... · pdf...

global competences

sub competences

bifm current competences

intercultural competences

evaluating the impact of virtual reality-based training on...