evidence of academic literacies · 2013-01-18 · to consider what we mean by academic literacy i...

National Center for Research on Evaluation, Standards, & Student Testing | Article | January 2013 1

Introduction_______________________________________________________________________________________________________________________

Let me begin with the de!nition of academic literacy suggested by our hosts, the World-

Class Instructional Design and Assessment at the University of Wisconsin (WIDA). Academic

literacy is…

“…the ability of students to make use of diverse resources to negotiate meaning in ways that accomplish the students’ purpose in communicating in academic settings and are valued within a 21st century community.”

To consider what we mean by academic literacy I want to highlight an experience that I am

sure many of us have had – our reading takes on different forms depending on our identities as readers. For example, as an English major I remember reading Jane Austen with a view to

analyzing the plot and character, with the goal of demonstrating my learning in seminars or

by answering test questions. When reading for such purposes, my metacognitive machine was

ticking all the time. Today, when I read Jane Austen for fun, I become immersed in the narrative

without the academic requirements of analysis. In the !rst instance, I was reading as a student –

in the second, I was reading as a person of leisure. My identity had changed.

When students engage in academic literacy, their identity is one of a learner. In academic

settings, learners view academic literacy through a variety of different perspectives that I believe

are implicit in the WIDA de!nition. When we talk about “academic literacy” we really mean

“academic literacies,” which vary depending on the particular identity of the reader.

Evidence of Academic LiteraciesMargaret Heritage | CRESST/UCLA

Evidence of Academic Literacies | Margaret Heritage 2

For example, consider the differences of meaning in the following tasks (see appendix for the tasks):

1) Analyze dialogue in The Merchant of Venice to examine how it contributes to an understanding of Shylock’s character.

2) Describe the proof for the theorem that the diagonals of a rhombus are perpendicular and bisect each other.

3) Follow instructions and fold a sheet of paper into the form of a cat.

4) Use a software’s “Help” function to create a new variable in a statistical program.

Each activity involves an entirely distinct form of literacy in the simplest sense of making meaning from text. Each involves an entirely different type of engagement between the text and the real world: for example, imagining an interaction and the disclosure of character and the motive within it; grasping a proof of a real-world circumstance that is true under all contingencies; following instructions with paper in hand to produce a model; following instructions to create an entity within a statistical program. Very different cognitive resources are engaged in the reading and understanding of each activity. The technical speci!cs of the language(s) involved are dramatically distinct. Even talking about each task requires a grasp of the speci!c conventions

of their use. This last distinction I refer to as the pragmatic competence of academic language.

As illustrated in Figure 1, pragmatic competence combines with academic language resources and cognitive resources to form the three dimensions of academic literacies.

Academic Literacies________________________________________________________

Academic Language ResourcesAcademic literacies require a variety of academic language resources. In the example of the rhombus proof I referred to earlier, a reader must understand how the expressions are connected to form a logical chain of argument as he or she makes sense of the proof. Quite a different form of connectivity is involved in reading Shakespeare where the sentences are connected to reveal an underlying pattern of motivation, which is itself disclosed in dialogue. In the example of creating a cat from a series of instructions, the reader has to translate each sentence in a manual operation with paper in hand to !nd out what the instructions mean in practice, while keeping in mind the total gestalt of the model and how the instructions !t (see Livingstone, 2008 for an illuminating discussion of this process). In the statistical program instructions, the representation of the variables requires very substantial knowledge of the nature and type of variables as well as how typed instructions apply to variables of different kinds.

ACADEMICLANGUAGERESOURCES

PRAGMATICCOMPETENCE

COGNITIVERESOURCES

Figure 1. Dimensions of academic literacies.


Each of these activities involves different conventions for reading, writing, and talking about text, different word-to-world relationships, different understandings of the purpose for which the text is being presented, and different conventions for text analysis and use. Countless and distinct forms of cognitive and pragmatic competencies underwrite all of these.

Cognitive ResourcesAcademic literacies require the development and deployment of a wide range of cognitive resources that are anchored in language, visuals, diagrams, and manipulations. Such cognitive resources will vary by domain. For example, among those in the texts we have reviewed so far are:

dialogue that conveys action and motives (Shylock’s motives);

lines, angles, intersections, and the nature of a proof itself (theorem proof);

to the manual construction of a model and inference, based on the developing model when the instructions are suf!ciently precise and when they are not, as well as what to do about it (paper folding to form a cat); and

represents in terms of these actions, the types of variables, and how the subordinate instructions relate to other pages of the help manual (using “Help” function in a statistical program).

Pragmatic CompetencePerhaps you recall as a child representing your home address beginning with the number and street and ending in the universe. My address would be 227 Tilden Ave, Los Angeles, California, The United States of America, The World, The Universe. When asked where you live, and since all of the address is true, what is the appropriate answer? Knowing the appropriate answer is pragmatic competence. In other words, knowing which part of one’s address to provide in response to the question involves knowing what to include and what to exclude. Similarly, knowing to use the words “I see you” in an academic essay rather than the message texting format, “ICU,” is pragmatic

competence, as is knowing that in a court of law one refers to law enforcement personnel as “police of!cers” rather than “cops.”

Classroom Context________________________________________________________

Consider what a high school science teacher is looking for in some of the ways students are asked to talk and write in his class (Heritage, 2013). The students are learning about how a mutation (a change in the DNA base sequence) may or may not alter the function of a protein. In the course of their learning, the teacher wants the students to write a paragraph explaining how changing the base sequence of DNA may or may not cause a functional change. To support their writing the teacher plans to ask students these questions:

1) How might a change of one base in the DNA sequence change the amino acid sequence in a polypeptide?

2) How could this affect the cell structure or activity?

3) Can you explain why a change in the DNA base sequence might NOT alter a cell’s structure or activity?

These questions require students to employ academic language, cognitive resources, and pragmatic competencies. In terms of pragmatic competencies, the students must understand that the questions require oral explanations, which they develop into a written paragraph. Pragmatic competence requires knowing how to talk about the subject in a way that makes sense to the teacher and to other students. Students need to know what to include and exclude in the explanation in terms of content and language in both oral and written forms. The students’ cognitive resources involve cause and effect reasoning as well as evidence-based justi!cation. Student language resources include the ability to use language to express their knowledge, adequate structures and technical vocabulary to convey their knowledge, and the ability to write connected discourse. Together, these speci!c academic language and cognitive resources and pragmatic competencies comprise one form of academic literacy in a science class.


Classroom Communities of PracticeEtienne Wenger noted that participating in a playground clique or in a work team is both an action and a form of belonging, shaping not only what the participants do, but also who they are and how they interpret what they do (Wenger, 1998). Participation as a learner in a classroom community of practice is identical. The student doesn’t just acquire subject matter knowledge, but must acquire the ability to act as a learner among other learners. As

“

“

Whatever the purpose of gathering evidence of academic literacy, the evidence should address academic literacies in the full range of contexts in which they are employed and the full range of academic language, cognitive resources and pragmatic competencies that inhere in each context.

Jerome Bruner once observed, students do not just simply learn about, they also learn to be (Bruner, 1996). As Bruner's comment implies, acting as a learner within a community of practice shapes the student's identity as a learner. This identity, in turn, is re"ected in the perceptions and behaviors of other class members (Seeley Brown & Duguid, 2001). Greeno and colleagues suggest that learning environments and activities need to be organized so that students can acquire basic skills, knowledge, and conceptual understanding not in isolation, but in ways that contribute both to the students’ self-identity as a strong learner as well as effective participants in the social practices of their school learning communities (Greeno & the Middle School Mathematics Through Application Project Group, 1998).

Students develop contextualized competencies that incorporate the learning practices and social interactional capacities that are valued within the speci!c community (Refer again to the WIDA de!nition). Social interactional capacities involve learning how to participate in the discourse practices that organize discussions, including building on the ideas of others, explaining one’s thinking in ways that make sense to listeners, listening for the sense in others’ thinking, providing evaluative and constructive feedback, asking questions, and resolving differences of opinion. Each capacity requires academic language,

Evidence of Academic Literacy Development________________________________________________________

Purpose of EvidenceEvidence of any kind of learning is collected through some form of assessment. The collection of evidence has two fundamental purposes: 1) to provide information on students’ current levels of achievement to the present time, and 2) to inform the future steps that teachers need to take in classrooms to ensure that students make progress toward desired outcomes. While a range of stakeholders may use evidence of current achievement to inform decisions and actions, the evidence collected for the second purpose is used to inform the decisions and actions of teachers and their students on an ongoing basis. In this paper, I refer to the !rst purpose as requiring summative evidence and the second purpose as requiring formative evidence.

Whatever the purpose of gathering evidence of academic literacy, the evidence should address academic literacies in the full range of contexts in which they are employed as well as the full range of academic language, cognitive resources, and pragmatic competencies inherent in each context. For example, the English teacher who engages students in reading, discussing, and writing about Shylock’s character and motivations needs evidence that

cognitive resources, and pragmatic competencies. The needed elements from those three will vary according to the learning context. Being an effective participant in the social and interactional practices of learning communities is another academic literacy multiplicity that students need to develop. Acquiring this form of academic literacy is both a process and a goal of learning (cf. Dewey, 1902/1990). In what follows, I discuss how we might collect and use evidence of academic literacies.


the students understand the dialogue, can infer motive, and can demonstrate their understanding through their own conversations and writing. She will most likely want evidence that students are able to do the preceding (have they achieved the goal?), plus evidence that the students are in the process of learning how to make meaning and draw inferences about Shylock. For their decision-making purposes, administrators, researchers, and policy makers will likely be more concerned with evidence that students have achieved the goal.

The math teacher who asks students to read and discuss the rhombus proof needs evidence of the students’ knowledge of lines, angles, and intersections; their ability to make sense of the text; and to express their ideas orally. The science teacher wants evidence of students’ understanding of mutations expressed orally and in writing. Both teachers will also want evidence while the students are in the process of learning.

Each of the three teachers also wants evidence of students’ literacy in the social interactional practices of learning – ways in which students participate in classroom discourse practices. Other stakeholders, except for parents and perhaps researchers, will be less likely to want evidence of students’ interactional practices, although this might be an oversight, given how important these practices are in school and in work.

So how can we collect evidence of the range of student academic literacies that serve assessment purposes while providing students the opportunity to reveal their learning based on the three dimensions of academic literacies: academic language, cognitive resources, and pragmatic competence? Before addressing that question, I turn to the observations of two prominent scholars in the area of educational assessment who have recently reminded us of some of the shortcomings of current assessment practices.

With respect to summative assessment, Paul Black observed that “Systems of summative assessment based solely on external testing are un!t for a single purpose, given their inevitably restricted validity, the manner in which they exert pressures that undermine good practices in teaching and defeat their own stated aim of improving learning, while at the same time undermine the professional status of teachers, which is essential for the health of any educational system” (Black, 2013, p. 176). Lorrie Shepard, commenting on classroom summative and formative assessment, noted that to re"ect learning that matters “classroom summative measures, whether projects, portfolios – or tests – must be deeply grounded in subject-matter content and processes. And to support, deep learning, formative assessment must elicit student thinking and provide substantive insights rather than quantitative score reports” (Shepard, 2013, p. xix).

Taken together, Black and Shepard’s comments encourage assessment that focuses on and supports learning that matters.

Summative EvidenceReturning to the question of how evidence of academic literacies can be collected, whatever the purpose of the assessment, it should generate evidence in ways that are consistent with the contexts in which academic literacies occur in classrooms. Students need opportunities to read, write, and talk about topics, ideas, and theorems that require the resources and competencies of academic literacies across a range of contexts. Unfortunately, most current summative assessments administered by states and districts rely on multiple-choice items that address low levels of thinking and language (Hopefully, the federally funded next-generation assessments will depart from this current situation.). In Black’s comments above, he challenges their “restricted validity,” a charge that is

“

“

Unfortunately, most current summative assessments administered by states and districts rely on multiple-choice items that address low levels of thinking and language.


appropriately leveled at summative measures of academic literacies that comprise this limited content. What students need in order to demonstrate evidence of academic literacies are rich tasks, “deeply-grounded” in the literacy of the domain and through which language, cognition, and pragmatic competence are revealed. Consequently, a single measure of academic literacy will not suf!ce. Needed are multiple opportunities in a variety of contexts for students to engage in a broad range of academic literacies.

In this vein, it is worth noting the comments of Stanley, McCann, Gardner, Reynolds, and Wild (2009) who point to the more active role that teachers should play in summative assessments, including high-stakes tests.

“The teacher is increasingly being seen as the primary assessor in the most important aspects of assessment. The broadening of assessment is based on a view that there are aspects of learning that are important but cannot be adequately assessed by formal external tests. These aspects require human judgment to integrate the many elements of performance behaviors that are required in dealing with authentic assessment tasks” (p. 31).

Indeed, with regard to academic literacies, formal tests involve students in private settings. During these tests, students respond to whatever assessment is administered. However, these tests will not capture oral discourse or the interactional practices of the classroom through which meaning is made and understood, nor will it capture the ability of students to make use of what they read and write in dialogue with others. While rich, well-constructed tasks can yield worthwhile evidence with respect to reading and writing, the absence of oral evidence from authentic settings limits the validity of any inferences of academic literacies. Perhaps the collection of evidence of oral performance can be left to teachers. After all, other countries seem to manage to include teacher judgment in summative assessment and regard it as a positive element of the assessment system (e.g., Wyatt-Smith, Klenowski, & Gunn, 2010).

Formative EvidenceErickson (2007), refers to formative assessment as constant “taking stock” of learning by “paying !rst-hand observational attention to students during the ongoing course of instruction” (p. 187). As noted earlier, this form of assessment is for the purpose of informing teaching and

learning, and rests in the control of classroom teachers. Teachers can plan evidence gathering into the “rhythm of the instruction, built-in as part of the constant interaction that is essential to ensure that the teacher and the learner are mutually and closely involved to a common purpose” (Black, Wilson &Yao, 2011, p. 98).

When teachers clearly understand academic literacies and the indicators of successful performance, they can plan various ways to collect evidence during teaching and learning. For example, in the high school science classroom discussed earlier, the teacher had decided on a set of questions that would provide evidence of students’ understanding of the DNA concepts. Student responses to these questions would also provide evidence of academic literacies noted earlier in the example, if the teacher was aware of the skills involved in academic literacy during the science lesson.

Grif!n (2007) argues that humans can only provide evidence of cognitive and affective learning through four observable actions: what they say, write, make, or do. These behaviors act as indicators of an underlying learning construct, and are the ways in which the observer can infer learning. I would add that humans (students) can only provide evidence of academic literacies by what they say, write, make, or do. The teacher’s role is to construct or devise ways to elicit responses from students that reveal

“

“When teachers clearly understand academic literacies and the indicators of successful performance, they can plan various ways to collect evidence during teaching and learning.


their current academic literacy in a range of content areas. Asking questions, engaging in dialogue, requiring students to read and talk about text, and write about what they have read, investigated, or discussed, are all sources of evidence a teacher can use to gain insights into the dimensions of academic literacies.

Evidence generated by the teacher has the advantage of being contextualized in everyday classroom learning tasks. To the degree that the classroom learning tasks promote the development of academic language, cognitive abilities and pragmatic competence, the evidence gathered will be grounded in the core learning of academic literacies. The teacher’s job is to interpret student performance revealed by the interactions and tasks to make judgments about the status of their learning and decide on what to do in order to advance that learning.

One !nal note. Whatever methods the various stakeholders use to collect evidence of academic literacies, the process should contribute to students’ strong self-identities as

learners. Tasks or tests that don’t re"ect important learning goals of academic literacies and that fail to engage students in worthwhile activities may diminish students’ identities as competent and capable learners. If students are to reveal the academic literacy competence to their teacher and peers in the public activity of classroom discourse, then they need to feel secure that mistakes, incomplete ideas, confusions, or “"awed”# language will not be treated as occasions for ridicule or sanctions, but rather as sources of new learning for the individual and the class as a whole. How teachers respond to students provides critical cues for how students respond to each other. ________________________________________________________

This article is based on a Margaret Heritage presentation and paper from the World-Class Instructional Design and Assessment meeting at the Wisconsin Center for Educational Research October 9-12, 2012. She may be reached at [email protected].

¹ I have “borrowed” this term from Guadalupe Valdez.


References__________________________________________________________________________________________________________________

Black, P. (2013). Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy. In McMillan, James (Ed.). (2013). SAGE Handbook of Research on Classroom Assessment, 167-197. Thousand Oaks, CA: SAGE.

Black, P., Wilson, M., & Yao, S. (2011). “Road maps for learning: A guide to the navigation of learning progressions,” Measurement: Interdisciplinary Research and Perspectives 9, (2–3), 98.

Bruner, J. (1996). The Culture of Education. Cambridge, MA: Harvard University Press.

Dewey, J. (1902/1990), The Child and the Curriculum. Chicago, IL: University of Chicago Press.

Erickson, F. (2007). Some thoughts on “proximal” formative assessment of student learning. Yearbook of the National Society for the Study of Education, 106, 186–216.

Greeno, J.G. and the Middle School Mathematics Through Application Project Group. (1998). “The situativity of knowing, learning, and research,” American Psychologist 53 (1), 5-26.

Grif!n, P. (2007). The comfort of competence and the uncertainty of assessment. Studies in Educational Evaluation, 33, 87–99.

Heritage, M. (2013). Formative Assessment: A Process of Inquiry and Action. Harvard Education Press: Cambridge, MA.

Livingston, E. (2008). Ethnographies of Reason. Ashgate Publishing Limited: Aldershot, England.

Seely Brown, J., & Duguid, P. (2001). “Knowledge and Organization: A Social-Practice Perspective,” Organization Science 12 (2), 198-213.

Shepard, L. (2013). Foreword In McMillan, James (Ed.). (2013). SAGE Handbook of Research on Classroom Assessment, xix –xxiii. Thousand Oaks, CA: SAGE.

Stanley, G., McCann, R., Gardner, J., Reynolds, L., & Wild, I. (2009). Review of teacher assessment: What works best and issues for development. Oxford, UK: Oxford University Centre for Educational Development.

Wenger, E. (1998). Communities of Practice: Learning Meaning and Identity. New York, NY: Cambridge University Press.

Wyatt-Smith, C., Klenowski, V., & Gunn, S. (2010). The centrality of teachers’ judgment practice in assessment: A study of standards in moderation. Assessment in Education: Principle, Policy & Practice, 17(1), 59–75.


Appendix__________________________________________________________________________________________________________________

Reading Task 1

SHYLOCK Antonio is a good man.

BASSANIO Have you heard any imputation to the contrary?

SHYLOCK Oh, no, no, no, no: my meaning in saying he is a good man is to have you understand me that he is suf!cient. Yet his

means are in supposition: he hath an argosy bound to Tripolis, another to the Indies; I understand moreover, upon the Rialto, he hath

a third at Mexico, a fourth for England, and other ventures he hath, squandered abroad. But ships are but boards, sailors but men:

there be land-rats and water-rats, water-thieves and land-thieves, I mean pirates, and then there is the peril of waters, winds and

rocks. The man is, notwithstanding, suf!cient. Three thousand ducats; I think I may take his bond.

BASSANIO Be assured you may.

SHYLOCK I will be assured I may; and, that I may be assur’d, I will bethink me. May I speak with Antonio?


Reading Task 2


Reading Task 3


Reading Task 4

AIf a type is speci!ed, the result returned by =exp must be string or numeric according to whether type is string or numeric. If str is

speci!ed, a str# variable is created, where # is the smallest string that will hold the result. See [D] egen for extensions to generate.

Replace changes thecontents of an existing variable. Because replace alters data, the command cannot be abbreviated. Set type speci!es

the default storage type assigned to new variables (such as those created by generate) when the storage type is not explicitly speci!ed.

Options: Nopromote prevents replace from promoting the variable type to accommodate the change. For instance, consider a variable

stored as an integer type (byte, int, or long), and assume that you replace some values with nonintegers. By default, replace changes the

variable type to a "oating point ("oat or double) and thus correctly stores the changed values. Similarly, replace promotes byte and int

variables to longer integers (int and long) if the replacement value is an integer but is too large in absolute value for the current storage

type. Replace promotes strings to longer strings. Nopromote prevents replace from doing this; instead, the replacement values are

truncated to !t the current storage type.

evidence of academic literacies · 2013-01-18 · to consider what we mean by academic literacy i...

Documents