journal of digital humanities

7/30/2019 Journal of Digital Humanities

1/108

VOL . 1 NO. 4 FAL L 2012

DANIEL J. COHEN, EDITOR

JOAN FRAGASZY TROYANO, EDITOR

SASHA HOFFMAN, ASSOCIATE EDITOR

JERI WIERINGA, ASSOCIATE EDITOR

ISSN 2165-6673

CC-BY3.0

ROY ROSENZWEIG CENTER FOR HISTORY AND NEW MEDIA

GEORGE MASON UNIVERSITY


2/108

With this fourth issue we wrap up the first year of the Journal of

Digital Humanities, and with it, our first twelve months of attempting

to find and promote digital scholarship from the open web using a

system of layered review. The importance of assessment and the

scholarly vetting process around digital scholarship has been foremost

in our minds, as it has in the minds of many others this year. As digital

humanities continues to grow and as more scholars and disciplines

become invested in its methods and results, institutions and scholars

increasingly have been debating how to maintain academic rigor while

accepting new genres and the openness that the web promotes.

Some scholarly societies, universities and colleges, and departments

have called for a redefinition or at least an expansion of what is

considered creditable scholarship. There have been scattered initial

attempts to understand how digital scholarship might be better

assessed, but the editors ofJDHfelt, and many of our readers agreed,

that there was not a single place to go for a comprehensive overview of

proposals, guidelines, and experiences. We attempt to provide a single

location here, with an issue and living bibliography that will grow as

additional examples are published across the web.

We begin with an identification of the scope of the problem, some

reasons for the difficulty assessing digital scholarship, and a call for

action. First, Sheila Cavanagh explains how the expectations of

traditional scholarship and the breadth of support required for

successful and creative scholarly and pedagogical projects restrict

younger scholars. Bethany Nowviskie suggests that modifying outdated

modes of peer review to recognize and credit the intellectual and

technical labor of the many participants who produce ambitious and

collaborative projects will positively influence the evolution of

scholarship writ large. The collaboratively-written Call to RedefineHistorical Scholarship in the Digital Turn, led by Alex Galarza, Jason

Heppler, and Douglas Seefeldt, was submitted as a formal request for

the American Historical Association to recognize and address these

particular issues.

In the next section, practitioners from across the academy and the

world offer their perspectives on assessment and evaluation. Todd

Presner, Geoffrey Rockwell, and Laura Mandell propose evaluation

criteria specifically for tenure and promotion. James Smithies details atypology of digital humanities projects to ensure proper

evaluation. Shannon Christine Mattern advises that the same detailed

criteria used to evaluate multimodal work in her classroom can serve

the larger academy. Zach Coble offers the view from the library, which

is the home of many collaborators and creators of digital humanities

projects. Finally, Sheila Brennan suggests that we further highlight the

intellectual goals and achievements of digital humanities projects

i

Closing the EvaluationGap
http://journalofdigitalhumanities.org/1-4/a-call-to-redefine-historical-scholarship-in-the-digital-turn/http://journalofdigitalhumanities.org/1-4/how-to-evaluate-digital-scholarship-by-todd-presner/http://journalofdigitalhumanities.org/1-4/a-call-to-redefine-historical-scholarship-in-the-digital-turn/http://journalofdigitalhumanities.org/1-4/let-the-grant-do-the-talking-by-sheila-brennan/http://journalofdigitalhumanities.org/1-4/let-the-grant-do-the-talking-by-sheila-brennan/http://journalofdigitalhumanities.org/1-4/evaluating-digital-humanities-work-guidelines-for-librarians-by-zach-coble/http://journalofdigitalhumanities.org/1-4/evaluating-digital-humanities-work-guidelines-for-librarians-by-zach-coble/http://journalofdigitalhumanities.org/1-4/evaluating-multimodal-work-revisited-by-shannon-mattern/http://journalofdigitalhumanities.org/1-4/evaluating-multimodal-work-revisited-by-shannon-mattern/http://journalofdigitalhumanities.org/1-4/evaluating-scholarly-digital-outputs-by-james-smithies/http://journalofdigitalhumanities.org/1-4/evaluating-scholarly-digital-outputs-by-james-smithies/http://journalofdigitalhumanities.org/1-4/promotion-and-tenure-for-digital-scholarship-by-laura-mandell/http://journalofdigitalhumanities.org/1-4/promotion-and-tenure-for-digital-scholarship-by-laura-mandell/http://journalofdigitalhumanities.org/1-4/short-guide-to-evaluation-of-digital-work-by-geoffrey-rockwell/http://journalofdigitalhumanities.org/1-4/short-guide-to-evaluation-of-digital-work-by-geoffrey-rockwell/http://journalofdigitalhumanities.org/1-4/how-to-evaluate-digital-scholarship-by-todd-presner/http://journalofdigitalhumanities.org/1-4/how-to-evaluate-digital-scholarship-by-todd-presner/http://journalofdigitalhumanities.org/1-4/how-to-evaluate-digital-scholarship-by-todd-presner/http://journalofdigitalhumanities.org/1-4/how-to-evaluate-digital-scholarship-by-todd-presner/http://journalofdigitalhumanities.org/1-4/a-call-to-redefine-historical-scholarship-in-the-digital-turn/http://journalofdigitalhumanities.org/1-4/a-call-to-redefine-historical-scholarship-in-the-digital-turn/http://journalofdigitalhumanities.org/1-4/a-call-to-redefine-historical-scholarship-in-the-digital-turn/http://journalofdigitalhumanities.org/1-4/a-call-to-redefine-historical-scholarship-in-the-digital-turn/http://journalofdigitalhumanities.org/1-4/evaluating-collaborative-digital-scholarship-by-bethany-nowviskie/http://journalofdigitalhumanities.org/1-4/evaluating-collaborative-digital-scholarship-by-bethany-nowviskie/http://journalofdigitalhumanities.org/1-4/living-in-a-digital-world-by-sheila-cavanagh/http://journalofdigitalhumanities.org/1-4/living-in-a-digital-world-by-sheila-cavanagh/


3/108

declared, but perhaps buried, in administrative documents and

reports.

Several practitioners then offer their personal experiences with

evaluation and assessment to help others in this unchartered

territory. Mark Sample explains the approach to digital scholarship he

used in his tenure and promotion case, while Katherine D.

Harris offers her tenure and promotion statements as a resource for

others. Finally, students at the new digital humanities program at

University College Corkremind us that evaluation ultimately is meant

to encourage conversation, so practitioners need to be involved directly

in the definition of any standards.

Already some organizations and scholars have produced good

beginning guidel ines for assessment. The Modern Language

Association in particular has solicited in-depth discussions among its

membership and outside scholars who have long worked in new media

on how to assess new forms of scholarship involving digital media and

technology. Other institutions, such as the Organization of American

Historians and the National Council on Public History, have made

some entreaties to broaden the definition of scholarly communication

that will require fleshing out in the years to come. We have re produced

some of that content at the end of this issue. We end the issue with

abibliographyof additional suggested readings on the evaluation andassessment of digital humanities work.

For the broadest possible understanding of the assessment of digital

scholarship, we asked the communityto help us find good case studies,

personal accounts, and departmental and institutional efforts. This

issue brings the best of these into one place that we can continue to

update as other guidelines and experiences are shared. We hope that

scholars in digital humanities and related fields will be able to point to

this volume of the Journal of Digital Humanities as a resource for

digital assessment and a starting place for further conversations.

Daniel J. Cohen and Joan Fragaszy Troyano, Editors

ii
http://digitalhumanitiesnow.org/2012/11/cfp-evaluation-and-assessment-of-digital-humanities-scholarship/http://journalofdigitalhumanities.org/1-4/bibliography/http://journalofdigitalhumanities.org/1-4/tenure-promotion-and-the-publicly-engaged-historian/http://journalofdigitalhumanities.org/1-4/documenting-a-new-media-case-evaluation-wiki-from-the-mla/http://journalofdigitalhumanities.org/1-4/documenting-a-new-media-case-evaluation-wiki-from-the-mla/http://journalofdigitalhumanities.org/1-4/guidelines-for-evaluating-work-in-digital-humanities-and-digital-media-from-the-mla/http://journalofdigitalhumanities.org/1-4/evaluating-digital-scholarship-experiences-in-new-programmes-at-an-irish-university/http://journalofdigitalhumanities.org/1-4/evaluating-digital-scholarship-experiences-in-new-programmes-at-an-irish-university/http://journalofdigitalhumanities.org/1-4/explaining-digital-humanities-in-promotion-documents-by-katherine-harris/http://journalofdigitalhumanities.org/1-4/tenure-as-a-risk-taking-venture-by-mark-sample/http://journalofdigitalhumanities.org/1-4/tenure-as-a-risk-taking-venture-by-mark-sample/http://digitalhumanitiesnow.org/2012/11/cfp-evaluation-and-assessment-of-digital-humanities-scholarship/http://digitalhumanitiesnow.org/2012/11/cfp-evaluation-and-assessment-of-digital-humanities-scholarship/http://journalofdigitalhumanities.org/1-4/bibliography/http://journalofdigitalhumanities.org/1-4/bibliography/http://journalofdigitalhumanities.org/1-4/tenure-promotion-and-the-publicly-engaged-historian/http://journalofdigitalhumanities.org/1-4/tenure-promotion-and-the-publicly-engaged-historian/http://journalofdigitalhumanities.org/1-4/tenure-promotion-and-the-publicly-engaged-historian/http://journalofdigitalhumanities.org/1-4/tenure-promotion-and-the-publicly-engaged-historian/http://journalofdigitalhumanities.org/1-4/documenting-a-new-media-case-evaluation-wiki-from-the-mla/http://journalofdigitalhumanities.org/1-4/documenting-a-new-media-case-evaluation-wiki-from-the-mla/http://journalofdigitalhumanities.org/1-4/documenting-a-new-media-case-evaluation-wiki-from-the-mla/http://journalofdigitalhumanities.org/1-4/documenting-a-new-media-case-evaluation-wiki-from-the-mla/http://journalofdigitalhumanities.org/1-4/guidelines-for-evaluating-work-in-digital-humanities-and-digital-media-from-the-mla/http://journalofdigitalhumanities.org/1-4/guidelines-for-evaluating-work-in-digital-humanities-and-digital-media-from-the-mla/http://journalofdigitalhumanities.org/1-4/guidelines-for-evaluating-work-in-digital-humanities-and-digital-media-from-the-mla/http://journalofdigitalhumanities.org/1-4/guidelines-for-evaluating-work-in-digital-humanities-and-digital-media-from-the-mla/http://journalofdigitalhumanities.org/1-4/evaluating-digital-scholarship-experiences-in-new-programmes-at-an-irish-university/http://journalofdigitalhumanities.org/1-4/evaluating-digital-scholarship-experiences-in-new-programmes-at-an-irish-university/http://journalofdigitalhumanities.org/1-4/evaluating-digital-scholarship-experiences-in-new-programmes-at-an-irish-university/http://journalofdigitalhumanities.org/1-4/evaluating-digital-scholarship-experiences-in-new-programmes-at-an-irish-university/http://journalofdigitalhumanities.org/1-4/explaining-digital-humanities-in-promotion-documents-by-katherine-harris/http://journalofdigitalhumanities.org/1-4/explaining-digital-humanities-in-promotion-documents-by-katherine-harris/http://journalofdigitalhumanities.org/1-4/explaining-digital-humanities-in-promotion-documents-by-katherine-harris/http://journalofdigitalhumanities.org/1-4/explaining-digital-humanities-in-promotion-documents-by-katherine-harris/http://journalofdigitalhumanities.org/1-4/tenure-as-a-risk-taking-venture-by-mark-sample/http://journalofdigitalhumanities.org/1-4/tenure-as-a-risk-taking-venture-by-mark-sample/


4/108

The Problem Stated

Living in a Digital World: Rethinking Peer Review, Collaboration, and Open Access

Sheila Cavanagh . . . . . . . . . . . . . . . . . . . . . 4Evaluating Collaborative Digital Scholarship (or, Where Credit is Due)

Bethany Nowviskie . . . . . . . . . . . . . . . . . . . . . 17A Call to Redefine Historical Scholarship in the Digital Turn

Alex Galarza, Jason Heppler, and Doug Seefeldt . . . . . . . . . . . . . . . 31


5/108

SHEILACAVANAGH

Living in a Digital World:Rethinking Peer Review,Collaboration, and OpenAccess

Its no secret that times are tough for scholars in the humanities. Jobs

are scarce, resources are stretched, and institutions of tertiary

education are facing untold challenges. Those of us fortunate enough

to hold tenured positions at financially stable colleges and universities

may be the last faculty to enjoy such comparative privilege. The future

shape of the academy is hard to predict, except to acknowledge that it

is unlikely to remain static. Our profession is being rapidly

reconfigured, but many changes are not happening quickly enough. In

the realm of the digital, for example, entrenched traditional standards

of assessment, support, and recognition still fail to encourage the kind

of exciting new research that keeps our disciplines vibrant.

Wh il e so me or ga ni za ti on s, such as th e Modern Language

As so ci at io n (MLA) and the National Endowment for the

Humanities (NEH), have made significant efforts to address the need

for national dialogues about germane topics, numerous faculty

members, department chairs, deans, and others involved in the faculty

reward system continue not to understand the shifting parameters of

research, teaching, and service that have been instigated by the digital

revolution. Many of these individuals, in fact, remain unaware of their

ignorance. Those who do not work in digital realms themselves often

unwittingly contribute to an environment that impedes intellectual

innovation. Despite the pressing need for reconfigured standards of

evaluation and new approaches to mentoring, many of those holding

the power to address this situation do not recognize the issues at stake.

Failure to redress current circumstances would have serious

consequences for the humanities. Fields such as those promoted by

this journal are especially vulnerable, since they often do not attract

the widespread attention needed to survive in difficult times. It is

important, therefore, for administrators and faculty at all levels to

respond to the particular ways that conventional academic evaluative

and mentoring models often inadvertently impede important new

work.

In a letter to the MLA, past President Sidonie Smith notes:

Experimenting with new media stimulates new habits of mind and

enhanced cultures of collegiality. Future faculty members in the

modern languages and literatures will require flexible and

improvisational habits and collaborative skills to bring their

scholarship to fruition.(2) Smiths remarks reflect the evolving reality

of todays academy. As we struggle with shrinking resources and other

changes to our academic environment, her words demand careful

consideration.

As director of the Emory Women Writers Resource Project (EWWRP)

since 1995, editor of the Spenser Review (now online rather than

produced in print) and co-director, with Dr. Kevin Quarmby, of

theWorld Shakespeare Project (WSP), I have a personal investment in

the success of the digital humanities. As a tenured, full professor,

however, my career is not unduly influenced by the status of my digital

work. During prev ious promotion deliberations, my digita l

contributionspredominantly focused on the study of early modern4
http://www.neh.gov/http://www.worldshakespeareproject.org/http://www.neh.gov/http://www.mla.org/http://www.neh.gov/http://www.neh.gov/http://www.mla.org/http://www.neh.gov/http://www.mla.org/http://www.worldshakespeareproject.org/http://www.worldshakespeareproject.org/http://www.spenserreview.org/http://www.spenserreview.org/http://womenwriters.library.emory.edu/http://womenwriters.library.emory.edu/http://www.neh.gov/http://www.neh.gov/http://www.neh.gov/http://www.neh.gov/http://www.mla.org/http://www.mla.org/http://www.mla.org/http://www.mla.org/


6/108

womenwere ignored. At this point, I enjoy the opportunity to pursue

such avenues without worrying about employment security. While my

professional reputation and compensation are still influenced by my

scholarly productivity, whether digital or in print, such pressures are

obviously less critical than those facing graduate students and junior

scholars.

As collaborator and mentor to several such members of the academic

community, I would like to draw from my experience with their

projects to illustrate some of the ways that scholarship is changing and

to suggest the kinds of concurrent alterations needed in our

assessment and mentoring practices. As my title suggests, I believe that

our traditional conceptualization of peer review, the humanities

continuing hesitance to support collaborative ventures, and our

common inability to mentor junior colleagues appropriately remainprimary obstacles to the kind of digital humanities work that can help

our disciplines flourish even during difficult times. While open access

in todays academic discourse generally signifies freely available digital

materials, I would like to expand that term in order to examine the

obstacles impeding junior scholars seeking open access to digital

creation.

One of the changes I want to highlight is the way that peer review has

evolved fairly quietly during the expansion of digital scholarship andpedagogy. Even though some scholars, such as Kathleen Fitzpatrick,

are addressing the need for new models of peer review, recognition of

the ways that this process has already been transformed in the digital

realm remains limited. The 2010 Center for Studies in Higher

Education (hereafter cited as Berkeley Report)comments astutely on

the conventional role of peer review in the academy:

Among the reasons peer review persists to such a degree in the academy

is that, when tied to the venue of a publication, it is an efficient indicator

of the quality, relevance, and likely impact of a piece of scholarship. Peer

review strongly influences reputation and opportunities. (Harley, et al 21)

These observations, like many of those presented in this document,

contain considerable wisdom. Nevertheless, our understanding of peer

review could use some reconsideration in light of the distinctive

qualities and conditions associated with digital humanities. The statusof peer review has shifted, but there have not been sufficient

conversations about the implications of those changes. While there is

some understanding that digital work demands new configurations of

review, there is still insufficient awareness that these processes have

already been changed in substantial ways. Nevertheless, some scholars,

such as Steve Anderson and Tara McPherson, emphasize the dangers

accompanying such shortsightedness:

Yet we resist such change at our peril. In a moment when universities andgovernments in the United States and abroad seem intent on shrinking

the humanities and on interrogating their value, digital media offer an

avenue to reinvigorate our scholarship and to communicate it in

compelling new ways. This capacity of the digital to present work to a

broader audience means that our work can circulate in many forms, in

different affective registers, and in richer dialogues. (149)

The work of many scholars would benefit from such changes. As

market forces and other non-intellectual considerations reduce

opportunities for scholarly exchange in smaller humanistic fields, such

as womens writing, electronic media offers great promise that should

be supported, rather than constrained.

As an example of important alterations already silently occurring in the

peer review process, I would like to draw attention to the work of Dr.

Melanie Doherty, a junior humanist at Wesleyan College in Macon,

Georgia, a college serving a socioeconomically diverse population of

women. A few months ago, Dr. Doherty sent me (as Director of the

5


7/108

EWWRP) an email, asking if I would be interested in a digital archive

project that she was creating with Sybil McNeil in the Wesleyan

Library. Her message offered an overview of this endeavor:

As Wesleyan College celebrates its 175th anniversary this year as the first

college in the world chartered to grant degrees to women, our library has

begun to digitally archive student writings and artifacts from themid-19th through 20th century. Wesleyan holds a wealth of unique

materials discussing womens history in the South. These include student

writings, speeches from visiting dignitaries, and letters from notable

19th- and 20th-century feminists, as well as photos, clothing and

artworks that span the schools rich past from the 1840s to the present

day. These artifacts detail invaluable information about the lives of

women in the antebellum South through the world wars, womens

suffrage, and Civil Rights movements, all documenting the achievements

of women with fascinating insight into their daily lives.

Your collection has already featured work from some notable Wesleyan

alums, including Loula Kendall Rogers, and we have much more material

that would be relevant to the Emory archive. As sister college to Emory,

and as your institution also celebrates its 175th anniversary this year, we

thought this might present a great opportunity to collaborate.

Intrigued by the project, I met with Dr. Doherty several times in person

and over Skype. I also gathered a group of relevant local library and

technological personnel, so that we could all discuss whether and how

Wesleyans archival efforts could be supported by Emory. As these

conversations evolved, several key issues emerged regarding the

atypical nature of peer review and collaboration in digital humanities.

The academic review aspect of this undertaking illustrates a

noteworthy, but under-recognized shift in the professional trajectories

of junior scholars involved in digital humanities. Dr. Doherty

approached me, in part, because Wesleyan does not have sufficient

server capacity to house any archives that she and McNeil are able to

produce. In addition, Emory (and Georgia Tech, another potential

partner) possesses a range of technological equipment and expertise

that Wesleyan cannot replicate. Facing such obstacles is a standard

feature in modern digital scholarship, as the Berkeley Report makes

clear:

humanists are seldom able to pay for extensive support out of personal

research funds and many voiced the need for in-house (i.e.,institutional) technical support for individual research projects. Libraries

are often on the front lines of supporting these faculty with their research

and publication needs. For example, the library is assumed, in many

cases, to be the locus of support for archiving, curation, and

dissemination of scholarly output. (27)

Accordingly, Doherty proposed that the EWWRP might house the

Wesleyan archive as a distinctive collection among the others currently

comprising this digital enterprise. This prospect made immediate

sense to me. The Wesleyan archive appears to be of significant

academic interest, and I believe strongly in supporting the efforts of

talented junior scholars, particularly when they are working on

projects involving Womens Studies and digital humanities.

6Emorys Center for Interactive Teaching


8/108

Over the years, I have been able to offer tangible, moral, and advocacy

support to a number of less-established scholars, both male and

female, who have grown interested in these fields. In this instance,

however, while I could fulfill the crucial role of facilitator, I could not

provide the level of authorization that Dr. Doherty would need in order

to submit the strong grant proposal she was trying to create. AlthoughI direct the EWWRP, I do not own the digital space it inhabits. I work

closely with colleagues in Academic Technology and the Woodruff

Library, but I make no decisions regarding the allocation of their time,

expertise, or priorities. At the same time, these colleagues typically

have some ability to determine where to devote their attention, but

generally lack the authority to decide independently what kinds of

projects they will support in their capacity as Emory employees. As I

note elsewhere (Cavanagh 5), this situation contrasts dramatically with

my experience of starting the EWWRP. In the mid-nineties (a lifetimeago in digital chronology), faculty and librarians at Emory faced

comparatively few similar constraints. It was an era of fledgling digital

exploration. Those of us experimenting in these realms formed

partnerships with limited official interference. We were not required to

justify our efforts very often, in part because relatively few people were

paying much attention. Dr. Chuck Spornick and Dr. Alice Hickcox in

the Lewis H. Beck Center for Electronic Collections and Services were

charged with supporting faculty with digital endeavors. Fortunately for

me, they were eager to become engaged with the EWWRP and have

remained valuable collaborators ever since.

Today, however, there are a number of competing needs and priorities

that potential Emory partners, such as Dr. Hickcox in the Beck Center

and Dr. Stewart Varner of Emorys Digital Scholarly Commons, need to

address before they can offer ongoing participation in any project. Like

other units of the university, Woodruff Library has its own Strategic

Plan detailing its official ambitions, goals, and priorities. Within the

Library and in various divisions of Information Technology, numerous

business plans and other germane documents identify the kinds of

endeavors that will further these aims. As readers of this journal

probably know all too well, women writers and womens history are not

likely to figure prominently in typical university technological vision

statements. There may or may not be active opposition to this kind ofacademic focus, but faculty in these fields cannot presume that

everyone will recognize the value of such projects. The individuals

making decisions about technological resourcesare often not scholars

themselves, while even those who offer both scholarly and technical

expertise are likely to come from disparate fields. Accordingly, while

review remains, traditional conceptualizations of peer recede.

This common situation leads to the largely unseen shift in the kind of

review current digital scholars encounter. In traditional printscholarship, faculty face peer review much later in the trajectory of

their research. They might, at some point, apply for a grant, but many

humanistic scholars complete their projects successfully with

appropriate access to relevant library collections and sufficient time to

devote to their research. Faculty at more affluent institutions often

access more financial resources and more amenable teaching loads

than their colleagues with less comfortable circumstances, but

everyone is eligible to apply for grants and fellowships from

organizations like the NEH or the ACLS. According to conventionalwisdom, moreover, scholars are often best situated to receive such

grant support if they apply after their work is largely completed.

Applications written when the relevant research has already been done

are said to provide more compelling accounts indicating the worthiness

of the project. I have never seen non-anecdotal evidence confirming

this common belief, but the premise carries considerable logical merit.

7


9/108

Digital work, such as Dr. Dohertys, cannot be created under

comparable circumstances, however. As detailed above, the successful

implementation of her plans for a digital archive requires a

significantly different review process. She cannot present a finished or

nearly completed project for evaluation; she needs approval from

varied sources before she can even proceed past the conceptual stage ofher endeavor. Numerous people from several institutions need to agree

that her idea holds merit and fits within existing, non-scholarly

priorities, before she can move forward with it. This situation reflects

todays norm. As the MLA Guidelines for Evaluating Work with Digital

Media in the Modern Languages suggests, digital scholars invariably

work with a range of project collaborators:

Humanists are not only adopting new technologies but are also actively

collaborating with technical experts in fields such as image processing,

document encoding, and information science. Academic work in digital

media should be evaluated in the light of these rapidly changing

institutional and professional contexts, and departments should

recognize that some traditional notions of scholarship, teaching, and

service are being redefined.

Notably, however, this now common reconfiguration of faculty work

makes it difficult to characterize the procedures Doherty followed as

involving traditional peer review. Unlike the blind evaluative

procedures followed in conventional promotion, tenure, and grantreviews, Dr. Doherty needed to approach people openly and directly.

She also required assistance in determining who to contact at potential

partner institutions, since such information can be impossible to

discern from university websites. In addition, the typical

conceptualization of what constitutes a peer becomes complicated in

these instances.

Since a digital project demands support outside the faculty of a given

institution, the work regularly requires authorization from those who

do not typically engage in faculty peer review. The necessary

evaluation, moreover, often includes serious consideration of factors

that have nothing to do with scholarly quality. Like the many university

presses that have eliminated monograph series or gone trade for

financial rather than intellectual reasons, those able to authorize

digital projects make decisions based on a broad range ofconsiderations that are distinct from elements key to promotion and

tenure discussions.

At a large university, for example, projects in the humanities may be

competing for funding and attention with proposals from diverse

professional schools. Resources might be allocated by individuals

without a particular commitment to the humanities or by those holding

any number of competing interests. Unlike a journal article, book

proposal, or grant application that is sent to an expert in a relevantfield, a digital decision can be made by people from a range of

positions, both academic and not, within a college or university. A

successful application may indicate scholarly value, but not

necessarily, just as a failed proposal may stop a scholar in his or her

tracks, but may not suggest that the idea was flawed.

Obviously, traditional scholarship also confronts the influence of

chance, mistake, or other arbitrary roadblocks, but the distinctive

situation facing scholars in digital humanities is not widelyacknowledged. While a scholar applying for a research grant from the

Folger Shakespeare Library does not generally face an applicant pool

containing faculty from Engineering, Business, or Law, faculty

pursuing digital support often do. The concept of open access,

therefore, which many academics currently perceive as a primary value

in digital production, exists in an environment that is far less open

than many scholars recognize. Successful projects may be

disseminated through the process termed open access, but that does

8
http://www.mla.org/guidelines_evaluation_digitalhttp://www.mla.org/guidelines_evaluation_digitalhttp://www.mla.org/guidelines_evaluation_digitalhttp://www.mla.org/guidelines_evaluation_digitalhttp://www.mla.org/guidelines_evaluation_digital


10/108

not mean that there is open access to developmental resources. In

reality, open access to the range of personnel and equipment needed

to bring a digital humanities project to fruition is rarely available.

For the purposes of this essay, I am not proposing a specific, one size

fits all response to these circumstances; rather, I am encouraging

faculty who hire, tenure, and mentor junior scholars to acknowledge

the complicated factors in the world of digital scholarship needing

attention. In addition to the under-recognized importance of non-peer

review in digital undertakings, for example, faculty often have

difficulty identifying appropriate experts to participate in more

traditional peer review processes. Could an Aphra Behn scholar with

no background in electronic media, for instance, provide appropriate

evaluation of a digital Behn resource? Would a digital humanist with

no familiarity with Behn be a more or less qualified assessor? At whatstage is peer review needed? As the MLA Guidelines indicate, Faculty

members who work with digital media should have their work

evaluated by persons knowledgeable about the use of these media in

the candidates field. An appropriate level of familiarity with digital

work is particularly important for outside reviewers, since many

faculty members have not been part of informed discussions about how

to evaluate digital scholarship. In a hiring discussion at Emory

recently, for example, a normally astute faculty member with little

digital background remarked that since anyone can post anything onthe web, departments should only evaluate items published

electronically after standard peer review processes.

While this perspective is understandable, it demonstrates a common

inability to consider the need for revised evaluative guidelines if we are

going to encourage innovative new scholarship. Self-publishing on

the web, for instance, does not correspond to traditional print self-

publishing as closely as many non-digitally savvy faculty members

believe. The web certainly can serve as an electronic vanity press, but it

can also facilitate rapid and revisable dissemination of important

scholarly material. Not recognizing the differences between

appropriate traditional and digital review is likely to hurt scholarship,

as Kathleen Fitzpatrick notes:

Imposing traditional methods of peer review on digital publishing might

help a transition to digital publishing in the short term, enabling more

traditionally-minded scholars to see electronic and print scholarship as

equivalent in value, but it will hobble us in the long term, as we employ

outdated methods in a public space that operates under radically

different systems of authorization. (9)

As Fitzpatrick suggests, a reimagining of peer review will provide a

crucial step toward needed academic progress. Traditional peer review

often does not meet the needs of electronic production. In an article ona related topic (Cavanagh 10) I recently described the significant

scholarly achievement demonstrated by my colleague Harry

Rusches Shakespeares World websites, even though Professor

Rusches work did not undergo standard peer review. Since Professor

Rusche began his impressive archive long after he received tenure, he

was not impeded by the paucity of evaluative bodies available to offer

peer review for projects such as his that are created without grant

funding.

Only a few groups, such as NINES (Networked Infrastructure for

Nineteenth-Century Electronic Scholarship) and its sister

group 18thConnect, provide this type of external review for digital

work within their subject areas. In addition, NINES is partnering with

the NEH to formulate detailed review guidelines for projects emerging

across the digital humanities horizon (Wheeles). Nevertheless, the

field of evaluation for digital scholarship is still largely under

9
http://www.mla.org/guidelines_evaluation_digitalhttp://www.mla.org/guidelines_evaluation_digitalhttp://www.nines.org/http://institutes.nines.org/docs/http://institutes.nines.org/docs/http://www.18thconnect.org/http://www.18thconnect.org/http://www.nines.org/http://www.nines.org/http://shakespeare.emory.edu/http://shakespeare.emory.edu/http://www.mla.org/guidelines_evaluation_digitalhttp://www.mla.org/guidelines_evaluation_digital


11/108

development. In the meantime, both junior and senior faculty

members continue to expand their digital projects.

While Ruschea full professorcan devote considerable attention to

his acclaimed collection of Shakespearean postcards, however, an

untenured scholar would be taking a significant risk by following this

example. Although the quality of such work can be assessed throughappropriate criteria, many institutions have not addressed what

standards might be applicable for their hiring or promotion and tenure

processes. Access to the opportunity of creating digital work is

currently denied to many untenured scholars, therefore. Written

guidelines for digital assessment rarely exist and many tenured faculty

members remain unable, unwilling, or blind to the need to adapt

current promotion criteria to digital scholarship.

Not surprisingly, the privilege of undertaking digital scholarship thus

often falls to those who have already received tenure through

traditional channels. Mentoring practices tend to reinforce this

pattern. According to the Berkeley Report, for example:

The advice given to pre-tenure scholars was quite consistent across all

fields: focus on publishing in the right venues and avoid spending too

much time on public engagement, committee work, writing op-ed pieces,

developing websites, blogging, and other non-traditional forms of

electronic dissemination . . . (10)

Scholars on the tenure track accordingly often resist such risky

avenues, given the considerable pressures associated with the pre-

tenure probationary period. Academics with even less employment

stability, such as graduate students and other non-tenure track

scholars, face additional challenges that also need more serious

attention than they currently receive. In the next section of this

discussion, I would like to highlight the work of three such scholars,

graduate students Amy Elkins and Catherine Doubler at Emory, and

my collaborator, Dr. Kevin Quarmby, a recent Ph.D. who teaches in

London. None of these promising scholars currently hold tenure track

positions. They are all involved in exciting digital projects, however,

that demonstrate the short-sightedness of pushing scholars to

postpone such endeavors until after tenure, while underscoring the

significant scholarly benefits possible if faculty and administrators

more actively encouraged electronic scholarship of many kinds.

Amy Elkins won the 2011 South Atlantic Modern Language Association

Graduate Student Essay Prize for her essay, Cross-Cultural Kodak:

Snapshot Aesthetics in the Fiction of Virginia Woolf. This print essay

is forthcoming inSouth Atlantic Review. As this accolade suggests, Ms.

Elkins is a talented literary scholar, whose graduate career shows great

potential. Fortunately, she is not restricting herself to the print

domain, however. One of her scholarly projects involves the creation ofan intriguing digital archive that draws from several institutional

collections. She describes the project in a recent email:

For some time Ive been working on creating a digital archive of

thePotters Wheel, a manuscript magazine created by Sara Teasdale and

a group of women artists and writers (they called themselves The Potters)

in St. Louis from 1904-1907. Ive located all of the extant manuscripts in

special collections libraries, and Ive been working to get those libraries to

digitize their holdings so that I can get the page images on an Omeka site.

I envision a scholarly resource, as well as a teaching resource for a rangeof scholars across the disciplines.

Elkins details the trajectory of her digital creation in terms that

resonate with many who enter this field:

Working on a DH [digital humanities] project has put me in touch with a

whole range of amazing scholars. Ive opened up lines of communication

with professors who have an interest in DH such as yourself, a wonderful

Teasdale scholar who is totally behind the archive, other graduate

students working on the intersection of visual art, book history, and the10
http://samla.memberclicks.net/sarhttp://samla.memberclicks.net/sar


12/108

digital, and a network of DH enthusiasts on Twitter. . . Also, the staff at

Yales Beinecke Library has been tremendously helpful. . . The

hindrances: Not all libraries are equipped to do high quality digitization

or they dont want to use their manpower helping someone from another

institution. DH is truly collaborative, which means that you have to rely

on other people to get the balls rolling.

As Elkins is discovering, the collaborative efforts involved in digital

work can be both exhilarating and frustrating. They also require a

different skill set than was needed by many of the tenured faculty who

are mentoring upcoming generations of students. Traditional print

scholarship often leads to intellectual exchanges at conferences and

elsewhere, but it does not demandcooperation as frequently as digital

humanities does. While there may be a range of personality types

represented among humanities academics, the conventional image of a

scholar working in comparative isolation corresponds to the largelysolitary process that has led to many scholarly articles and

monographs in print. One might eagerly share ideas with colleagues

over coffee at the Newberry Library, for instance, but the rest of an

archival scholars day is likely to be spent predominantly with the

librarys holdings. Conversations with knowledgeable colleagues may

be valuable in this model, but they are generally not imperative for the

mere existence of a project. In digital humanities, however, it is a rare

scholar who is able to actualize an entire project without substantial

contributions by a host of technologists, librarians, and others whoseknowledge complements that provided by the scholar(s) envisioning an

electronic product.

These necessary partnerships offer further complications to issues

involving access. Clearly, collaborative work has a different history in

the humanities than in the sciences and conventional reward

structures in humanistic disciplines do not always easily accommodate

mutual efforts. Although a few humanists, such as Lisa Ede and Andrea

Lunsford, address the challenges and benefits of collaborative work,

humanistic fields have generally not caught up with such work.

Procedures for determining how to assess individual contributions to

joint endeavors can be developed, but most humanities departments

have yet to initiate such discussions in any serious or systematic way.

Given the widely recognized transformation within traditional printpublication outlets, humanities scholars cannot afford to postpone

such vital discussions any longer. Newer scholars need to produce

work within current practical restraints. Senior faculty who assess this

scholarship and who hire and mentor this cohort are irresponsible if

they do not acquire the knowledge they need in order to bring

promotion and tenure criteria into alignment with technological,

material, and philosophical changes in the intellectual marketplace.

Standards do not need to be lowered, just shifted. Senior faculty mustrecognize, for instance, that many common contemporary scholarly

practices, such as collaboration, can no longer be perceived as aberrant

or unworthy of credit. In addition, as the MLA guidelines for

evaluating electronic scholarship suggests, credit may need to be

allocated unconventionally:

Institutions should also take care to grant appropriate credit to faculty

members for technology projects in teaching, research, and service, while

recognizing that because many projects cross the boundaries between

these traditional areas, faculty members should receive proportionate

credit in more than one relevant area for their intellectual

work. (Guidelines)

Digital scholarship is transforming our professional lives and none of

us will benefit by ignoring or resisting the challenges introduced by

these new formats and modes of thinking. Noting the importance of

such academic reconfigurations, the Berkeley Reportsuggests that: As

faculty continue to innovate and pursue new avenues in their research,

11


13/108

both the technical and human infrastructure will have to evolve with

the ever-shifting needs of scholars (iii).

Concurrently, the professoriate will also need to expand the range of

topics and media that are welcomed into scholarly conversations. As a

graduate student in the 1980s, I was warned not to undertake

scholarship on women writers until I had tenure. Similar cautions wereoffered to many contemporaries with scholarly interests in other fields

deemed professionally risky. Over the years, the kinds of scholarship

prompting such suspicion may change, but a pattern of resistance to

certain topics of inquiry recurs. As digital options broaden the types of

presentation models available to scholars, multimedia presentations

also arouse both caution and suspicion. Senior gatekeepers thereby

stand in the way of vibrant modes of innovation that may keep the

humanities alive.

Catherine Doublers work demonstrates how limiting such intellectual

restrictions can be. Her self-designated second book project concerns

the work of the controversial anti-Stratfordian Delia Bacon.

Understandably viewed skeptically by the Shakespearean

establishment, long wearied of spurious claims against the Bard of

Avon, Bacon is the kind of figure junior scholars are traditionally

being warned against investigating. Doubler, however, is expanding

her expertise in Bacons fascinating intellectual legacy with itssurprising connections to todays digital world, while she completes

her dissertation on Renaissance drama and becomes adept with

electronic media. As a result, Doubler is creating a tangible scholarly

product while exploring intriguing questions about the relationship

between theoretical issues emerging through modern media and those

raised by earlier intellectuals such as Bacon. At the moment, Doubler is

working on digital editions of Bacons three novellas, The Tales of the

Puritans. As she describes this undertaking, Doubler highlights the

unexpected theoretical issues emerging through this digitization effort:

I thought that representing Bacons life and work in digital venues

could fittingly highlight her own interest in literature and technology.

As part of this electronic process, Doubler has been learning TEI (Text

Encoding Initiative) mark-up, which she finds intersects significantly

with Bacons work:

I have had to make use of two systems of codes when looking at The Tales

of the Puritans: the first concerns itself specifically with literary meaning

while the second takes a less logocentric view in order to make the novel

legible in an online format. As such, I would like to use my experiences of

putting Bacon into code to reflect on Bacons own obsessions over the

concepts of ciphers and secret Languages.

While Doublers investigation of Bacons life and works and her

translation of these novellas into digital format are still in embryonicform, the questions emerging make it clear that the theories and

practices accompanying modern technology can i lluminate such earlier

texts in fruitful ways. Whether or not Delia Bacon proves to be a more

promising figure of study than previous Shakespeareans have thought,

the connection between nineteenth- and twenty-first-century

technological codes opens exciting new realms of study. Working

digitally in this way can make such work available, bypassing non-

qualitative concerns that often stall print publication. This kind of

intellectual risk-taking leads to lively and productive humanisticresearch. In contrast, keeping certain modes and topics off limits to

junior scholars impedes critical progress, just as demanding scholarly

isolation inhibits exploration of the intriguing questions new

technologies foster. Broadening the concept of open access, on the

other hand, to make a wider range of scholarly topics and practices

open, can invigorate the humanities during these times of

debilitating constraints. Expanding the communal impulse behind the

12


14/108

now commonly conceived understanding of open access could

transform humanistic research.

The kind of energizing intellectual and practical collaboration that Amy

Elkins has encountered in her Teasdale work and Melanie Doherty

developed for her Wesleyan proposal, moreover, illustrates the

importance of expanding and endorsing inter-institutional ventures aswell as intellectual partnership between individuals. Emorys strong

support of my collaboration with Dr. Kevin Quarmby in London

models the brand of forward thinking that can facilitate an array of

future scholarly initiatives, but it also demonstrates the value of shared

innovation and the benefit of deliberate cooperation between diverse

practical and intellectual goals. As ourWorld Shakespeare Project and

related endeavors have evolved, we have received practical support

from many faculty, staff, and administrators working far outside therealm of early modern drama. Their engagement remains vital to our

success, which is largely created through our distinctive, though

complementary skill sets. Dr. Quarmby acted professionally in

Londons West End for many years before completing his Ph.D. at

Kings College, London in 2008. He currently teaches for a number of

academic programs in London, and is actively seeking a permanent,

full-time, institutional affiliation. Although still living and working in

the United Kingdom, he has been named Distinguished Visiting

Scholar at Emorys Halle Institute for Global Learning andShakespeare Performance Specialist in Virtual Residence at

Emorys Center for Interactive Teaching. He has also received support

from Emorys Center for Faculty Development and Excellence. Clearly,

numerous individuals at Emory see advantages to the universitys

educational mission through the implementation of this transAtlantic

research and pedagogic partnership.

The many Emory educational and technological leaders who are

contributing to the work that Dr. Quarmby and I are jointly involved in

are not demonstrating blind altruism, however. They are not offering

technical support and other assistance simply from generosity. Rather,

they see our projects as mechanisms for testing new technological and

international opportunities that will benefit the University. They also

recognize the value to Emory of Dr. Quarmbys wide-ranging skills as

an academic and theatrical practitioner. Our first electronic

collaboration, which is ongoing, involves Dr. Quarmby leading acting

workshops with students enrolled in an upper-division Shakespeare

class. Uniformly praised by undergraduate participants, these sessions

enable us to explore the technological and pedagogical opportunities of

co-teaching simultaneously from two different countries while offering

students the unique perspective provided by a Shakespearean scholar,

who has also performed professionally at some of Britains mostrenowned venues, such as the Old Vic and the National Theatre. Alan

Cattier, Director of Academic Technology Services at Emory and an

impressive team at Emorys Center for Interactive Teaching, including

Wayne Morse, Chris Fearrington, and a cadre of dedicated graduate

students, recognize this electronic teaching project as a way to

experiment with videoconferencing in a setting where the students are

clearly well served. Rather than simply bringing in a guest lecturer for

a single class, this technological alliance makes it possible for Dr.

Quarmby to work individually with students and to partner with me inplanning and assessing assignments. We endeavor to create a

sustainable and scalable model of electronic collaboration that takes

advantage of technological advances responsibly. Emorys continuing

dedication to this project helps us accomplish those goals.

The World Shakespeare Project (WSP) has related, but not identical

aims. In addition to the technological partners mentioned above, the

WSP benefits from the enthusiastic support of Vice Provost Holli

13
http://www.ecit.emory.edu/http://www.halleinstitute.emory.edu/http://www.worldshakespeareproject.org/http://www.emorycfde.com/http://www.emorycfde.com/http://www.ecit.emory.edu/http://www.ecit.emory.edu/http://www.halleinstitute.emory.edu/http://www.halleinstitute.emory.edu/http://www.worldshakespeareproject.org/http://www.worldshakespeareproject.org/


15/108

Semetko, Director of the Halle Institute for Global Learning and of

Professor Steve Walton and his students in the Goizueta Business

School. The WSP links electronically with international Shakespearean

faculty and students in order to create and sustain Shakespearean

education and dialogue opportunities with populations that would not

be able to participate in such projects prior to modern technology.

Once again, Emorys significant assistance results from the innovative

vision of leaders such as Dr. Semetko and Dr. Walton, whose own areas

of professional expertise do not include Shakespeare. Nevertheless,

they appreciate the broader pedagogical and technological implications

of projects such as ours. Dr. Waltons students, for example, are

gaining relevant business experience by helping us craft a business

plan, while faculty across the campus benefit from our success with

communicating internationally despite disparities between time zones,cultural and educational differences, and widely variant technological

infrastructures. Dr. Quarmby and myself have a host of intellectual and

pedagogical goals to pursue through the WSP, but we can

simultaneously fulfill broader institutional needs without

compromising our own plans. This kind of mutual benefit does not

occur spontaneously, but can result from open discussions and

alertness to the needs of our domestic and international partners.

While Shakespearean drama falls outside the central academic scope ofthis journal, the WSP draws attention to a number of issues pertinent

to the intertwined topics of peer review, collaboration, and access that

affect scholars in all fields. As a long-time faculty member at a major

research university, I am fortunate enough to have an academic base

willing and able to support my own work and that of talented

colleagues, such as Melanie Doherty and Kevin Quarmby. As noted,

Wesleyan College does not possess the computer resources needed to

create and maintain its own digital archive, while Dr. Quarmby does

not currently have direct access in London to the range of technological

expertise available through Emory. While both of these scholars are

pursuing worthy academic projects, their institutional affiliations do

not provide the resources they need in order to complete their work.

Collaboration with a university like Emory is critical, therefore, since

this electronic work could not exist otherwise. With the library and

archival resources openly available in London, Dr. Quarmby could

produce his recent book, The Disguised Ruler in Shakespeare and His

Contemporaries (Ashgate, 2012), without this kind of institutional

backing. Serious digital work, in contrast, remains significantly less

possible for scholars working outside robust research institutions.

Such projects can be of substantial benefit to individual scholars and to

collaborating institutions, however, suggesting that there would be

great merit in wider support of such cooperative efforts.

Such inter-institutional cooperation and other collaborative models

can lead to projects that benefit all participants. Concurrently,

however, they highlight important changes in the shape of faculty work

that require more widespread attention. Senior humanists need to

recognize, for example, the vital role of evaluation outside traditional

peer review in the creation and sustenance of the kinds of the digital

products discussed here, if they are going to mentor their graduate

students and junior colleagues appropriately. In each instance outlined

above, most of the key personnel who determined whether or not theseprojects could continue were not faculty experts in the relevant field.

Although some of these individuals hold doctorates, they do not

generally fit the disciplinary profile typical departments would use

when choosing outside evaluators for these junior scholars tenure

reviews. Instead, they have been trained in a range of subjects, often

widely variant from the content specialty of the graduate students and

junior scholars approaching them for assistance.

14


16/108

While peer review remains important in the academy, senior faculty

would do well to mentor junior colleagues about the importance of

developing connections outside traditional disciplinary and faculty/

staff boundaries. Institutions could profitably offer training to graduate

students in the emerging entrepreneurial aspects of their professional

lives. Knowing who to contact in an institution for what kind of

support is a skill that not all humanists understand instinctively. Many

current senior faculty never needed to develop this ability during their

own careers. Increasingly, however, access to vital scholarly resources

is likely to depend upon developing expanded sets of skills, including

many that are not specifically intellectual. The partnership that Dr.

Quarmby and I are forging with the Goizueta Business School, for

example, and the many links we have created with international

institutions, do not result from anything we learned in graduate school,

but still illustrate the range of practical skills that are becomingnecessary for humanists to create successful careers in their

disciplines. While content knowledge will undoubtedly remain central,

it is unlikely to be sufficient for a scholar to thrive in a digital

environment.

My goal in this essay is to encourage conversations about significant

aspects of digital scholarship and pedagogy that have not yet surfaced

in the awareness of many key players in the intertwined process es of

mentoring, hiring, tenure, and promotion. Those who do not work inelectronic realms themselves need to acquire a clearer understanding

of the particular requirements of this rapidly expanding scholarly

domain. Access to the ability to create substantive digital work

emanates from markedly different sources than comparable access to

traditional scholarship and pedagogy. Once completed, the resulting

projects often do not easily fit conventional evaluative mechanisms.

Electronic media have become pervasive in all of our lives, just as

many institutions are facing severe financial constraints. These

concurrent realities bring an urgency to the issues addressed here that

contrast with the slow pace that often characterizes significant change

in higher education.

Originally published by Sheila Cavanagh at Interactive Journal forWomen in the Arts,March 2012.

Works Cited:

Anderson, Steve, and Tara McPherson. Engaging Digital Scholarship:

Thoughts on Evaluatin g Multimedia Scholarship.

Profession 2011: 149. Modern Language Association. Web. 23

Feb. 2012.

Cavanagh, Sheila T. Emory Women Writers Resource Project. EmoryUnivers i ty . 2006. Web. 23 Feb. 2012 . < h t t p : / /

womenwriters.library.emory.edu/>.

Cavanagh, Sheila T. How Does Your Archive Grow?: Academic Politics

and Economics in the Digital Age. Appositions: Studies in

Renaissance/Early Modern Literature and Culture. 4 (2011).

Web. 23 Feb. 2012.

Cavanagh, Sheila T., and Kevin A. Quarmby. World Shakespeare

Project: A Model for Live Shakespeare Interaction in the New

M e d i a W o r l d . N . p . n . d . W e b . 2 3 F e b . 2 0 1 2 .

.

Doherty, Melanie. Message to the Author. 9 Aug. 2011. E-mail.

Doubler, Catherine. Message to the Author. 24 Oct. 2011. E-mail.

Elkins, Amy. Message to the Author. 19 Oct. 2011. E-mail.

15
http://www.aphrabehn.org/ABO/?p=1755http://www.aphrabehn.org/ABO/?p=1755http://appositions.blogspot.com/2011/05/sheila-cavanagh-digital-archive.htmlhttp://appositions.blogspot.com/2011/05/sheila-cavanagh-digital-archive.htmlhttp://appositions.blogspot.com/2011/05/sheila-cavanagh-digital-archive.htmlhttp://womenwriters.library.emory.edu/http://womenwriters.library.emory.edu/http://www.aphrabehn.org/ABO/?p=1755http://www.aphrabehn.org/ABO/?p=1755http://www.worldshakespeareproject.org/http://www.worldshakespeareproject.org/http://appositions.blogspot.com/2011/05/sheila-cavanagh-digital-archive.htmlhttp://appositions.blogspot.com/2011/05/sheila-cavanagh-digital-archive.htmlhttp://appositions.blogspot.com/2011/05/sheila-cavanagh-digital-archive.htmlhttp://appositions.blogspot.com/2011/05/sheila-cavanagh-digital-archive.htmlhttp://womenwriters.library.emory.edu/http://womenwriters.library.emory.edu/http://womenwriters.library.emory.edu/http://womenwriters.library.emory.edu/http://www.aphrabehn.org/ABO/?p=1755http://www.aphrabehn.org/ABO/?p=1755http://www.aphrabehn.org/ABO/?p=1755http://www.aphrabehn.org/ABO/?p=1755


17/108

Fitzpatrick, Kathleen. Planned Obsolescence: Publishing, Technology,

and the Future of the Academy. New York: NYU Press, 2011: 9.

W e b . 2 3 F e b . 2 0 1 2 . < h t t p : / /

m e d i a c o m m o n s . f u t u r e o f t h e b o o k . o r g / m c p r e s s /

plannedobsolescence/>

Guidelines for Evaluating Work with Digital Media in the ModernLanguages. Modern Language Association. 2012. Web. 23 Feb.

2012. .

Harley, Diane, et al. Final Report: Assessing the Future Landscape of

Scholarly Communication: An Exploration of Faculty Values

and Needs in Seven Disciplines. CSHE 1.10 (Jan. 2010): 21.

Web. 23 Feb. 2012.

Rusche, Harry. Shakespeare Illustrated. Emory College English

Department. n.d. Web. 23 Feb. 2012. .

Smith, Sidonie. Beyond the Dissertation Monograph. Modern

Language Association Newsletter. 42.1 (Spring 2010): 2. Web.

23 Feb. 2012.

Whe ele s, Dana. NEH Summer Institute: Evaluating Digital

Scholarship. NINES: Nin eteenth-Century Scholarship

Online . 19 Oct. 2010. Web. 23 Feb. 2012. .

16
http://www.nines.org/news/?m=201010http://www.mla.org/nl_archivehttp://www.mla.org/nl_archivehttp://www.mla.org/nl_archivehttp://shakespeare.emory.edu/illustrated_index.cfmhttp://shakespeare.emory.edu/illustrated_index.cfmhttp://shakespeare.emory.edu/illustrated_index.cfmhttp://www.mla.org/guidelines_evaluation_digitalhttp://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://www.nines.org/news/?m=201010http://www.nines.org/news/?m=201010http://www.nines.org/news/?m=201010http://www.nines.org/news/?m=201010http://www.mla.org/nl_archivehttp://www.mla.org/nl_archivehttp://shakespeare.emory.edu/illustrated_index.cfmhttp://shakespeare.emory.edu/illustrated_index.cfmhttp://shakespeare.emory.edu/illustrated_index.cfmhttp://shakespeare.emory.edu/illustrated_index.cfmhttp://www.mla.org/guidelines_evaluation_digitalhttp://www.mla.org/guidelines_evaluation_digitalhttp://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/


18/108

BETHANYNOWVISKIE

Evaluating CollaborativeDigital Scholarship (or,

Where Credit Is Due)

This is the lightly edited text of a talk given at the 2011 NINES

Summer Institute, a National Endowment for the Humanities-funded

workshop on evaluating digital scholarship for purposes of tenure

and promotion, hosted by the Networked Infrastructure for

Nineteenth-Century Electronic Scholarship. It builds on a more

formal essay written for an open-access cluster of articles on the topic

in Profession, the journal of the Modern Language Association

(MLA). A pre-print of that essay was provided to NINES attendees in

advance of the Institute.

As youll divine from the image, Ill spend my time today addressing

human factors: framing collaboration within our overall picture for the

evaluation of digital scholarship. Ill pull several of the examples Ill

share with you from my contribution to theProfessioncluster that our

workshop organizers made available, and my argument will be familiar

to you from that piece as well. But I thought it might be useful to lay

these problems out in a plain way, in person, near the beginning of our

week together. Collaborative work is a major hallmark of digital

humanities practice, and yet it seems to be glossed over, often enough,

in conversations about tenure and promotion.

We can trace a good deal of that silence to a collective discomfort,

which much of my recent (service) work has been designed to expose

discomfort with the way that our institutional policies, like those that

govern ownership over intellectual property, codify status-based

divisions among knowledge workers of different stripes in our colleges

and universities. These issues divide digital humanities collaborators

in even the healthiest of projects, and well have time afterwards, I

hope, to talk about them.

But I want to offer a different observation now, more specific to the

process that scholars on tenure and promotion committees go through

in assessing readiness for advancement among their acknowledged

peers. My observation is that the tenure and promotion (T&P) processis a poor fit to good assessment (or even, really, to recognition) of

collaborative work, because it has evolved to focus too much on a

particular fiction. That fiction is one of final outputs in digital

scholarship.

In 2006, the MLAs task force on evaluating scholarship issued an

important report. It asserts the value of collaboration even in an

institutional situation where solitary scholarship, the paradigm of

one-authorone-work, is deeply embedded in the practices of17
http://www.mlajournals.org/doi/abs/10.1632/prof.2011.2011.1.169http://www.mlajournals.org/toc/prof/2011/1http://www.mlajournals.org/doi/abs/10.1632/prof.2011.2011.1.169http://www.nines.org/about/community/workshop.htmlhttp://www.nines.org/about/community/workshop.htmlhttp://www.mla.org/tenure_promotionhttp://www.mla.org/tenure_promotionhttp://nowviskie.org/2009/monopolies-of-invention/http://nowviskie.org/2009/monopolies-of-invention/http://www.mlajournals.org/toc/prof/2011/1http://www.mlajournals.org/toc/prof/2011/1http://www.mlajournals.org/toc/prof/2011/1http://www.mlajournals.org/toc/prof/2011/1http://www.mlajournals.org/doi/abs/10.1632/prof.2011.2011.1.169http://www.mlajournals.org/doi/abs/10.1632/prof.2011.2011.1.169http://www.mlajournals.org/doi/abs/10.1632/prof.2011.2011.1.169http://www.mlajournals.org/doi/abs/10.1632/prof.2011.2011.1.169http://www.nines.org/about/community/workshop.htmlhttp://www.nines.org/about/community/workshop.htmlhttp://www.nines.org/about/community/workshop.htmlhttp://www.nines.org/about/community/workshop.html


19/108

humanities scholarship, including the processes of evaluation for

tenure and promotion.

That sets a kind of charge for us, and Ill read the words of the task

force to you:

Opportunities to collaborate should be welcomed rather than treated withsuspicion because of traditional prejudices or the difficulty of assigning

credit. After all, academic disciplines in the sciences and social sciences

have worked out rigorous systems for evaluating articles with multiple

authors and research projects with multiple collaborators. We need to

devise a system of evaluation for collaborative work that is appropriate to

research in the humanities and that resolves questions of credit in our

discipline as in others. The guiding rule, once again, should be to

evaluate the quality of the results. (Report 5657)

I see this as a clear and unequivocal endorsement of the work for whichthe set ofpreconditions Ill offer you in a little bit intends to clear

ground. But I want to pick at that last sentence a little, and encourage

some wariness about the teleological thrust of the phrase, quality of

results.

The danger here (which many of you confirmed you see this

happening) is that T&P committees faced with the work of a digital

humanities scholar will instigate a search for print equivalencies

aiming to map every project that is presented to them, to some othercompleted, unary and generally privately-created object (like an article,

an edition, or a monograph). That mapping would be hard enough in

cases where it is actually appropriateand this week well be exploring

ways to identify those and make it easier to draw parallels. But I am

certain, if you look only for finished products and independent lines of

responsibility, you will meet with frustration in examining the

more interesting sorts of digital constructions. In examining, in other

words, precisely the sort of innovative work you wantto be presented

with. To make a print-equivalency match-up attempt across the board,

in every case, is to avoid a much harder activity, the activity I want to

argue is actually the new responsibility of tenure and promotion

committees. This is your responsibility to assess quality in digital

humanities worknotin terms of product or outputbut as embodied

in an evolving and continuous series of transformativeprocesses.

Many years ago, when we were devising an encoding scheme for a

project familiar to NINES attendees, the Rossetti Archive, two of our

primary sites for inquiry and knowledge representation were

theproduction history and the reception history of the Victorian texts

and images we were collecting and encoding. I find (as perhaps many

of you do) that I still locate scholarly and artistic work along these two

axes. In conversations about assessment, however, we are far too apt to

lose that particular plot. This is becauseproduction and reception have

been in some ways made new in new media (or at least a bit

unfamiliar), and also because theyve never been adequately embedded

again, as activities, not outcomesin our institutional methods for

quality control.

We have to start taking seriously the systems of production and of

reception in which digital scholarly objects and networks are

continuously made and remade. If we fail to do this, well shortchange

the work of faculty who experiment consciously with such fluiditybutworse: we will find ourselves in the dubious moral position of

overlooking other people, including many non-tenure-track scholars,

who make up those two systems.

Digital scholarship happens within complex networks of human

production. In some cases, these networks are simply heightened

versions of the relationships and codependencies which characterized

the book-and-journal trade; and in some cases they are truly

incommensurate with what came before. However you want to look at18
http://www.rossettiarchive.org/http://www.rossettiarchive.org/http://www.rossettiarchive.org/


20/108

them, its plain that systems of digital production require close and

meaningful human partnerships. These are partnerships that

individual scholars forge with programmers, sysadmins, students and

postdocs, creators and owners of content, designers, publishers,

archivists, digital preservationists, and other cultural heritage

professionals. In many cases, the institutional players have been there

for a long time, but collaboration, now, has been made personal

again (by virtue of the diversifying of skillsets) and is amplified in

degree through the experimental nature of much digital humanities

work. (This is an interesting observation to make, perhaps, about our

scholarly machine in the digital age. Despite all the focus on

cyberinfrastructure and scholarly workflows, were fashioning ever

closer, more intimate and personalized systems of production.)

To offer just one small example: compare the amountof conversation

about layout, typography, and jacket design a scholar typically has with

the publisher of a printed bookto the level of collaborative work and

intellectual partnership between a faculty member and a Web design

professional who (if theyre both doing their jobs well) work together to

embed and embody acts of scholarly interpretation in closely-crafted,

pitch-perfect, and utterly unique online user experiences.

But its not just that we (we evaluators, we tenure committees) fail to

appreciate collaboration on the production side. We neglect, too, to

consider the systems of reception in which digital archives and

interpretive works are situated. In many cases, the products of digital

scholarship are continually re-factored, remade, and extended by what

we call expert communities (sometimes reaching far beyond the

academy) which help to generate them and take them up. Audiences

become meaningful co-creators. And more: an understanding of

reception now has to include the manner in which digital work can be

placed simultaneously in multiple overlapping development and

publication contexts. Sometimes, perpetual beta is the point! Digital

scholarship is rarely if ever singular or done, and that complicates

immensely our notions of responsibility and authorship and readiness

for assessment.

So my contention is that the multivalent conditions in whichwe encounter and create digital work demonstrate just how much we

are impoverishing our tenure and promotion conversations when we

center them on objects that have been falsely divorced from their

networks of cooperative production and reception. Now, okay:

certainly, committees can and do confront situations in which

individual scholars have created digital works without explicit

assistance or with minimal collaborative action. But those have long

19

Some Scholar's Lab non-tenure track faculty and staff


21/108

been the edge cases of the digital humanitiesso why should our

evaluative practices assume that theyre the rule and not the exception?

Theres something deeper to this, though, and it has to do with the

academys taking, collectively, what is in effect a closed-down and

defensive stance toward the notion of authorship. As an impulse, it

certainly stems to the larger feeling of embattlement in our corner ofthe academy. But we must ask ourselves: do we really want to assert

the value and uniqueness of a scholars output by protecting an

outmoded and often patently incorrect vision of the solitary author? Is

that the best way to build and protect what we do, together? What kind

of favor do we think were doing the humanities, when we stylize

ourselves into insignificance in this particular way?

To get back to people, heres my fear: that were driving junior

scholars, who lack good models and are made conservative by complex

anxieties, toward two poor options. These are 1) dishonesty to self, and

2) dishonesty toward others. In the first case, we are putting them in a

position where they may choose to de-emphasize their own innovative

but collaborative work because they fear it will not fit the preconceived

notion of valid or significant scholarly contribution by a sole academic.

Thats dishonesty to self. The even nastier flip side is the second case:

causing them to elide, in the project descriptions they place in their

portfolios, the instrumental role played by othersby technical

partners and so-called non-academic co-creators.

Now, you might expect me to go straight for a mushy and obvious first

stepto argue today that we should work to increase our appreciation

for collaborative development practices in the digital humanities. It

makes sense that fostering an appreciationthat clarifying what

collaboration means in digital humanitiescould lead to a formal

recognition of the collective modes of authorship that digital work very

often implies. Unfortunately, we have to roll things back a bitand this

is why I used the word Preconditions in the title of

myProfession essay.

In too many cases (this is disheartening, but true) scholars and

scholarly teams need reminders that they must negotiate theexpression of shared credit at allmuch less credit that is articulated

in legible and regularized forms. By that I mean forms acceptable

within the differing professions and communities of practice from

which close collaborators on a digital humanities project may be

drawn.

We evaluate digital scholarship through a bootstrapped chain of

responsibilities. Professional societies and scholarly organizations set a

tone. Institutional policy-making groups define the local rules of20

Early-Career Scholars at the Scholars' Lab
http://mediacommons.futureofthebook.org/alt-ac/http://mediacommons.futureofthebook.org/alt-ac/http://mediacommons.futureofthebook.org/alt-ac/http://mediacommons.futureofthebook.org/alt-ac/


22/108

engagement. Tenure committees are plainly responsible for

educating themselves (they often forget this) about the nature of

collaborative work in the digital humanities, so that they may

adequately counsel candidates and fairly assess them. Scholars who

offer their work for evaluation are, in turn, responsible for making an

honest presentation of their unique contributions and of the

relationship they bear to the intellectual labor of others.

And digital humanities practitioners working

outside the ranks of the tenured and tenure-

track faculty have a role to play in these

conversations as well. Were talking here

about people like me and many of my

colleagues in the digital humanities world,

like the people I imagine partner with you at

your home institutions, and like some of the

folks who built NINES and 18th-Connect. We

are hybrid scholarly and technical

professionals subject to alternate, but equally

consequential (though often less protected)

mechanisms of assessment. We need you, the

tenured and tenure-track faculty, to support

us when we assert that credit must be given

where it is due. Ill talk in a little bit about aneventalso organized with National

Endowment for the Humanities (NEH)

supportthat took on exactly this issue, and

how making such assertions might hasten

the regularization of fair and productive

evaluative practice among tenure-track and

non-tenure-track digital humanities

practitioners alike.

But I have to stop to acknowledge that people on my side of that fence

(that is, humanities PhDs working as alternative academics off the

straight and narrow path to tenure) can sometimes be seen rolling

their eyes and wondering aloud whyyou guys remain so hung up ondefining individual (rather than your collective) self-worth. I have

observed a sotto voce countdown that often happens among

experienced digital humanists at panels on digital work at more

traditional humanities conferences: "Can we go ten whole minutes into

the Q&A without eating these particular worms?" My suspicion is that

many folks on the alt-ac track are where they are, not only because of

a congenital lack of patience, but because they are temperamentally

inclined to reject some concepts that other humanities scholars remain

21

Up by Their Bootstraps

Nobody Loves Me, Everybody Hates Me, I'm GonnaEat Some Worms
http://www.nines.org/about/community/workshop.htmlhttp://www.18thconnect.org/http://nowviskie.org/2012/lunaticks/http://nowviskie.org/2012/lunaticks/http://www.18thconnect.org/http://www.18thconnect.org/http://www.nines.org/about/community/workshop.htmlhttp://www.nines.org/about/community/workshop.html


23/108

tangled up in. And one of the most invidious of these is a tacit notion of

scholarly credit as a zero-sum game, which functions as an underlying

inhibitor to generous sharing.

But lets talk about this week. Wouldnt it be brilliant ifthis group,

imaginativeproduction,

enthusiasticpromotion,

and committedpreservation

of digital humanities work a shared and personal enterprise. Itll make

your scholarly work an enterprise in which, in the most granular

sense, named librarians, technologists, administrators, and

researchers will feel a private as well as professional stake. Youjust do

a better job, now and far into the future, with things that have your

name on them.

Maybe part of the reason it is so hard to latch onto the issue of proper

credit for diverse collaborators is that those collaborators are

represented by so many different professional societies and advocacy

groups. Lets check in with just a few. Ive found the most instructiveexamples in the field of public (which is often to say digital) history.

My favorite is a statement issued by a Working Group on Evaluating

Public History Scholarship, commissioned jointly by the American

Historical Association (AHA), the National Council on Public History,

and the Organization of American Historians (OAH). In 2010, they put

out something called Tenure, Promotion, and the Publicly Engaged

Academic Historian (PDF). This piece starts in same key I did today,

on the matter of process. It strongly endorses the AHAs Statement on

Standards of Professional Conduct, which defines scholarship as a

process, not a product, an understanding [they say] now common in

the profession. And it goes on:

The scholarly work of public historians involves the advancement,

integration, application, and transformation of knowledge. It differs from

traditional historical research not in method or in rigor but in the

venues in which it is presented and in the collaborative nature of its

creation. Public history scholarship, like all good historical scholarship, is

peer reviewed, but that review includes a broader and more diversegroup of peers, many from outside traditional academic departments,

working in museums, historic sites, and other sites of mediation between

scholars and the public. (Working Group 2)

Similarly, heres something from the MLAs 1996 report, Making

Faculty Work Visible:

As institutions develop their own means of assessment, they should

consider the wide range of activities that require faculty members

22

A zero-sum game?
http://www.mla.org/rep_facultyvishttp://www.mla.org/rep_facultyvishttp://ncph.org/cms/wp-content/uploads/Engaged-Historian.pdfhttp://ncph.org/cms/wp-content/uploads/Engaged-Historian.pdfhttp://www.mla.org/rep_facultyvishttp://www.mla.org/rep_facultyvishttp://www.mla.org/rep_facultyvishttp://www.mla.org/rep_facultyvishttp://ncph.org/cms/wp-content/uploads/Engaged-Historian.pdfhttp://ncph.org/cms/wp-content/uploads/Engaged-Historian.pdfhttp://ncph.org/cms/wp-content/uploads/Engaged-Historian.pdfhttp://ncph.org/cms/wp-content/uploads/

journal of digital humanities

Documents