books digitization

7/25/2019 Books Digitization

http://slidepdf.com/reader/full/books-digitization 1/20

he Million Book Digital

Library Project

Raj Reddy and Gloriana StClairCarnegie Mellon University

Pittsburgh, Pa. 15213

December 1, 2001

4122!"25#$

rr%cmu.e&u



Objective

The objective of this roject is

to create a free!to!read"searchable collection of one

#illion books" ri#arily in the

$nglish lang%age" available to

everyone over the &nternet' This

task is acco#lished by scanning

the books and inde(ing their f%ll

te(t' The te(t file is created"

)here ossible" thro%gh otical

character recognition' The res%lt)ill be a %ni*%e reso%rce

accessible to anyone in the )orld

+,(-(./0" )itho%t regard to

nationality or socioecono#ic

backgro%nd'



Tyical large high!school

libraries ho%se fe)er than .1"111

vol%#es' One #illion vol%#es isthe aro(i#ate si2e of the

co#bined libraries at Carnegie

Mellon 3niversity' The total

n%#ber of different titles inde(ed

in OCLC4s 5orldCat is abo%t ,6

#illion' One #illion books"

therefore" is #ore than the

holdings of any high!school"

e*%ivalent to the library at as%bstantial %niversity and a

significant fraction of all

available books'

$(ec%tive S%##ary

Creating a universal 'ree to rea&, &igital library containing over one million

scanne& boo(s, )ith o*tical character recognition )hen *ossible to su**ort 'ull te+t

searching, is the goal o' the million boo( &igital library *roect. -uch a resource )ill

lea& to the &emocratiation o' (no)le&ge by ma(ing available on the )eb, a uni/ue

library resource to scholars, stu&ents, an& citiens aroun& the )orl&. he availability

o' online search allo)s users to locate relevant in'ormation /uic(ly an& reliably thus

enhancing stu&ent )illingness an& success in their research en&eavors. his

24+$+3!5 resource )oul& also *rovi&e an e+cellent testbe& 'or language *rocessing

research in areas such as machine translation, summariation, intelligent in&e+ing,an& in'ormation mining.



*ortion o' the content )oul& inclu&e out o' co*yright, *re1#20 materials.

7 best boo(s8 'eature o' the *roect )oul& involve re/uesting *ermission to scan

titles in the core collection &evelo*ment tool Books for College Libraries.

*reliminary Carnegie Mellon University ibraries *ilot suggests that 22 o' the"0,000 titles might become available. urther, )hen "0 o' the million boo(s are

'inishe&, scholars )ill be recruite& to revie) collections in their &isci*lines an& to

select remaining boo(s o' im*ortance.

Mirroring the site at several locations )orl&)i&e )ill *rotect the integrity an& availability o'

the &ata. -everal mo&els 'or sustainability are being e+*lore& an& are &iscusse& in this re*ort.

Usability stu&ies )oul& also be con&ucte& to ensure that the materials are easy to locate, navigate,

an& use. **ro*riate meta&ata 'or navigation an& management )oul& also be create&.

ational -cience oun&ation is *rovi&ing 'un&ing 'or -canners, Com*uters, -ervers, an&

-o't)are. hese resources 'rom - are augmente& by almost t)enty to one since China an&

6n&ia )ill be *rovi&ing the necessary man*o)er 72,000 man years each, over a 'our year *erio&8,

as their contribution to this *roect, to assist in selection o' &ocuments, so't)are &evelo*ment an&

in &igitiing these materials. 6n&igenous Chinese an& 6n&ian materials )oul& 'orm a *ortion o' the

content scanne& as )oul& 9nglish language materials alrea&y resi&ent in those countries. 6n

a&&ition, U.-. libraries, *rimarily members o' the Digital ibrary e&eration, )oul& shi* materials

to be scanne& an& returne&.

&&' echnical

Descrition

9' Pri#ary Objective

he *rimary longterm obective is to ca*ture all boo(s in &igital 'ormat. -ome

believe such a tas( is im*ossible. hus as a 'irst ste* )e are *lanning to

&emonstrate the 'easibility by un&erta(ing to &igitie 1 million boo(s 7less than

1 o' all boo(s in all languages ever *ublishe&8 by 2005. :e believe such a

*roect has the *otential to change ho) e&ucation is con&ucte& in much o' the

)orl&. he *roect ho*es to create a universal &igital library 'ree to rea& any time

any )here by anyone.

9ach o' the million boo(s is scanne&. 6' it is in a language 'or )hich o*tical

character recognition so't)are is available, the te+t is converte& to ascii;unico&e

'ormat to allo) 'ull te+t search to gui&e stu&ents, scholars, an& citiens to the

relevant *ortions o' the )or(. -canner o*erators create meta&ata, base& on

e+isting cataloging recor&s 'or these boo(s an& ournals, to accom*any each boo(.



his *roect enhances research, learning, an& teaching by ma(ing a critical mass

o' scholarly in'ormation 'reely available to rea& online. 6t has been observe& that

the result )ill be li(e <annevar =ush4s Meme+. 6n a&&ition to its o)n in&e+es,

maor in&e+ers, such as >oogle )ill in&e+ it an& others, inclu&ing libraries *artici*ating in the *roect, )ill hy*erlin( to it.

secon&ary obective o' this *roect )ill be to *rovi&e a test be& that )ill

su**ort other researchers )ho are )or(ing on im*rove& scanning techni/ues,

im*rove& o*tical character recognition, an& im*rove& in&e+ing. he cor*us this

*roect creates )ill be at least ten times as large as any e+isting 'ree resource.

B' Pri#ary Benefit

Primary bene'it is to su**lement the 'ormal e&ucation system by ma(ing

(no)le&ge available to anyone )ho can rea& an& has access. ibraries have

*laye& a vital role in the a&vancement o' human society. -ocietal a&vance

&e*en&s on young *eo*le having access to boo(s via libraries an& other

means. :e e+*ect that ma(ing this uni/ue )eb resource available 'ree to

everyone in the U.-. an& aroun& the )orl& )ill lea& to a 'urther &emocratiation

o' access to (no)le&ge.

ibraries are unevenly &istribute& aroun& the )orl& an& )ithin countries. 6n the

U.-., the C9- -urvey note& that in 1##!, 3,40" o' 3,$#2 institutions o' higher e&ucation ha& libraries hol&ing "0!.$ million volumes. he 112 largest

university libraries in the Unite& -tates an& Cana&a each have at least 1." million

boo(s? they are members o' the ssociation 'or @esearch

ibraries. Massachusetts has about 25 million volumes? e) Aor( has about 31

million volumes, an& Cali'ornia has about 40 million volumes in their @

ibraries 7ssociation 'or @esearch ibraries, 1###;20008. Bther states, such as

orth an& -outh Da(ota, have no large libraries. 'e) large *ublic libraries

have several million volumes. o)ever, most unior colleges, high schools, an&

*ublic libraries have much smaller collections. Ma(ing this large (no)le&ge

re*ository )ith the convenience o' online access an& the bene'it o' )or& an&

*hrase 'ull te+t searching can revolutionie research at all levels o' e&ucation an&

give a muchnee&e& boost at minimal cost to our national e&ucational

in'rastructure.

-econ&ary bene'it Bnline search ma(es locating the relevant in'ormation insi&e

o' boo(s 'ar more reliable an& much easier. -tu&ent success in 'in&ing e+actly

)hat they see( )ill increase an& increase& success )ill enhance stu&ent

)illingness to *er'orm research in this large resource. C9- re*orts that "4

*ercent o' libraries aroun& the country are o*en bet)een !0 an& "0 hours a)ee(. his &igital library )oul& be o*en 24 hours a &ay, seven &ays a )ee(, an&



3!5 &ays a year 'or a total o' 1!" hours a )ee(, over t)ice the time most libraries

are o*en. More than one in&ivi&ual )ill be able to use the same boo( at the same

time. hus, *o*ular )or(s )ill not be chec(e& out an& thus unavailable to

others.

his millionboo( *roect )ill *ro&uce an e+tensive an& rich testbe& 'or use in

'urther te+tual language *rocessing research. 6t is ho*e& that at least 10,000

boo(s among the million )ill be available in more than one language, *rovi&ing a

(ey testing area 'or *roblems in e+am*le base& machine translation. 6n the last

stage o' the *roect, boo(s in multi*le languages )ill be revie)e& to ensure that

this testbe& 'eature is accom*lishe&.

Many believe that (no)le&ge is no) &oubling at the rate o' every t)o to three

years. Machine summariation, intelligent in&e+ing, an& in'ormation mining are

tools that )ill be nee&e& 'or in&ivi&uals to (ee* u* in their &isci*line )or(, in

their businesses, an& in their *ersonal interests. his large &igitiation *roect

)ill su**ort research in these areas.

C' Stat%s to Date

he *reliminary )or( &escribe& belo) has been use& to establish a *rotocol, to

select stan&ar&s to be use&, an& to a&&ress issues o' in&e+ing an&

retrieval. :or('lo) an& training *rograms to su**ort the larger *roect are being

&evelo*e&. =oth the content an& the mechanisms 'or using it )ill be ma&eavailable in o*en source co&e.

he ational -cience oun&ation4s 2000 6@ grant cycle *rovi&e& E500,000

'or e/ui*ment to begin a large *ilot. hat grant )ill allo) the *urchase o' 1"

Minolta boo( scanners to be locate& in 6n&ia an& China. -ome machines have

alrea&y been &e*loye& to begin the scanning *rocess. -trong &iscounts 'rom

Minolta have e+*an&e& the number o' machines that can be *urchase&. 9arlier

*ilot *roects, a 100boo( scanning *roect an& a 1000boo( scanning *roect,

that ai&e& in the selection o' the scanners an& the establishment o' *rocesses use&

are &escribe& more 'ully belo).

Chinese University Presi&ents, a Ministry o' 9&ucation o''icial, an& Chinese

ca&emy o' -ciences lea&ers visite& the U.-. to reach agreements an& to 'orm a

steering committee.

Dr. Michael es( an& Dr. -te*hen >ri''in 'rom - atten&e& the Carnegie

Mellon meeting an& also hoste& the Chinese &elegation at the ational -cience

oun&ation. Pro'essor P Aunhe, Presi&ent o' Fheiang University? Dr. >B

:en, De*uty Presi&ent o' the >ra&uate -chool, Chinese ca&emy o' -cience?

Pro'essor C6 uisheng, <ice Presi&ent o' =eiing University? Pro'essor UDongcheng, <ice Presi&ent o' singhua University? Pro'essor GU Fhong, <ice



Presi&ent o' u&an University? Pro'essor, F> Aibin, ssistant to the

Presi&ent, aning University? Mr. >UB Ginli, <ice >eneral Director, Ministry

o' 9&ucation o' China? Mr. C9 Hian*ing, <ice Director, -tate Planning

Commission o' China? an& Dr. ChingChih Chen o' -immons College

atten&e&. he ational -cience oun&ation 'un&e& this summit.

he 6n&ian university an& government o''icials are sche&ule& to visit on the

2!th o' May 2002 an& it is e+*ecte& that similar agreements )oul& be reache&.

U.-. Digital ibrary e&eration members met on ovember 15 an& 1!, 2001 to

)or( out the logistics o' selecting an& trans*orting materials 'rom U.-.

collections un&er a grant 'rom -. Drs. es( an& >ri''in )ere oine& in

Pittsburgh by re*resentatives 'rom BCC, the Center 'or @esearch ibraries, an&

collection &evelo*ment o''icers an& other librarians 'rom the ibrary o'

Congress, the University o' :ashington, the University o' Cali'ornia =er(eley,

-tan'or& University, University o' 6llinois, University o' Chicago, Penn -tate

University, an& the University o' Pittsburgh. he Digital ibrary e&eration4s

9+ecutive Director also atten&e& the meeting.

he collection &evelo*ment librarians &iscusse&

I Collection 'ocus to achieve a consensus on ho) to select the million boo(s to be

&igitie&.

I 6nvolvement o' outsi&e scholars in selection issues to consi&er ho) nonlibrarianscholars might *artici*ate in selection.

I Co*yright consi&erations to consi&er see(ing *ermission 'or a set o' in co*yright

7 best boo(s8, such as those in =oo(s 'or College ibraries.

I -tan&ar&s 'or the )or( to revie) the current Digital ibrary e&eration stan&ar&s )ith

a vie) to ra*i& a&o*tion.

I @egistry issues to move 'or)ar& )ith BCC in establishing a registry 'or boo(s

selecte&.

I Metho&s o' trans*ort to consi&er alternative means o' trans*ort an& return.

I iming to )eigh the a&vantages o' air containers an& sea containers.

I evel o' *artici*ation to &etermine minimum levels 'or contributors to the *roect.

I 6ncentives 'or *artici*ation to establish means o' recognition 'or contributions through

screen &is*lay an& co*ies o' the archives.

he outcome o' this meeting )ill result in a *lan 'or the selection an&

transmission o' almost a million boo(s to China an& 6n&ia over a multiyear *erio&

an& a *lan 'or assessing the success o' the *roect annually.

D' Technical 9roach



:' Database creation

Creating a scalable &atabase to su**ort this *roect is a relate& research

*ro*osal. Drs. Christos aloutsos, He''rey 9**inger, an& atalia

llamachi are submitting a *ro*osal to - to a&&ress these issues.heir globally &istribute& &atabase )ill a**ear to be a virtual central

&atabase 'rom any *lace aroun& the )orl&. Mirroring the &atabase in

several countries )ill ensure security an& availability.

he &atabase )ill house both an image 'ile an& a te+t 'ile at about 1020

megabytes *er boo(. he aggregate o' 20 terabytes )ill be a''or&able to

store because the costs o' storage continue to &ecline substantially. =y

2010, a terabyte o' storage is e+*ecte& to cost as little as E10.

+' Scanning

100 book pilot )o years ago, )e 'un&e& a *ilot e+*eriment to scan 100 boo(s so

that the *ractical &i''iculties o' a million boo( *roect coul& be assesse&. Carnegie

Mellon University ibraries 'aculty an& sta'' assiste& in the *ilot. he scanner o'

choice )as an ine+*ensive &u*le+ scanner that re/uire& the boo(s to be &isboun& so

that the *ages coul& be 'e& through in batches. :hile the economy an& s*ee& o'

this techni/ue )ere most attractive, several technical *roblems occurre&.

I he *ages ha& to be cut on all 'our e&ges 'or smooth 'ee&ing. he *roect

re/uire& the *urchase o' a E10,000 guillotine to accom*lish this. he guillotine

)as some)hat &angerous, re/uire& in&e*th training in use an& sa'ety, slo)e&

the *rocess, )as a *ublic relations nightmare 'or the library community, an&

obviate& the economy o' the ine+*ensive scanner.

I Dust, an inevitable accom*animent to ol&er boo(s, *rove& to be a 'ormi&able

o**onent. Dust cause& 're/uent amming an& subse/uent cleaning o' the

scanner. Pa*er 'i+atives )ere em*loye& to counteract the &ust. -*raying on

the 'i+ative slo)e& the *roect an& )as not entirely satis'actory.

t the en& o' the 'irst hun&re& boo(s, the scanner o*erators an& their

su*ervisor sought another a**roach.

1000 book project . =oo(s 200 through 1000 )ere scanne& using a Minolta

Bverhea& scanner. lthough this scanner )as 5 times more e+*ensive roll'ee&

&ouble si&e& scanner )e use&, it *rove& to be more reliable. =oo(s &i& not have to

be &isboun&. he image *rocessing so't)are 'or curvature correction, &es(e)ing,

&es*ec(ling an& cro**ing allo)s 'or thic( boo(s to be scanne& either 'lat or in an

angle& cra&le that re&uces )ear on the s*ine. horough training is re/uire& too*erate the scanner, but several &i''erent em*loyees )ere success'ully traine& to use



it &uring the *erio& o' the *roect. he results o' this 1000 boo( *roect can be

vie)e& at ))).ulib.cs.cmu.e&u un&er 1000 boo( *roect. his scanner an& the

*rocesses are the ones that are recommen&e& 'or the million boo( *roect. he

a&vantages o' the Minolta a**roach inclu&e

I Disbin&ing via a guillotine is not necessary.

I =oo(s can be reuse& in their original 'orm.

I Dust, thic( *a*er, an& long boo(s can be easily accommo&ate&.

I raining re/uirements are reasonable.

I 9/ui*ment is reliable.

3. Data Pro&uction

I =itonal images )ith a *i+el &e*th o' 1 bit*er*i+el )ere scanne& at a resolution

o' !00 &ots *er inch 7&*i8. 6mages store& as J6ntelJ 6 7agge& 6mage ile

ormat8 'iles, )ith the hea&er content s*eci'ie&. he com*ression algorithm

use& is 6U 7ormerly CC68 >rou* 4.

I 6 version 5.0 is acce*table. -ubect to testing, version !.0 7or later8 may

also be acce*table.

I 6nitialca*ture system inclu&es &ynamic threshol&ing or a similar 'eature to

ca*ture variability o' &ar(ness in the im*rint an& *ossibly &ar(er 7e.g., 'o+e&8

bac(groun&s 'rom &ecay. 6mages shoul& be as rea&able as the original *ages.

I Jy*icalJ or Je+*ecte&J &ata to be *rovi&e& 'or most 6 tags 7normally, the

&ata su**lie& by so't)are &e'ault settings8. s*eci'ication 'or the 6 hea&erto be *ro&uce& to inclu&e scanner technical in'ormation, 'ilename, an& other

&ata, but to be in no )ay a bur&en on the *ro&uction service.

I 6mages )ritten in se/uential or&er, )ith corres*on&ing ".3 'ile names, e.g.,

00000001.ti' as 'irst image in volume se/uence an& 00000341.ti' as 341st

image in volume se/uence

I <olumes to be *rovi&e& to Million =oo( Proect by libraries )ith uni/ue

i&enti'iers that con'orm to ".3 'ormat? images shoul& be in &irectories name&

)ith corres*on&ing i&enti'ier 7e.g., a('3435.001 as i&enti'ier 'or volume )ill

result in &irectory )ith same name, an& 00000001.ti' through 0000000.ti'

)ithin that &irectory8

I 6mages an& &irectories 7as s*eci'ie& above8 to be )ritten by Million =oo( Proect

to gol& CD@BM meeting agree& u*on s*eci'ications, an& using 6-B#!!0

'ormat.

I -(e) to be )ithin a s*eci'ie& range o' &egrees allo)e&.

4. B*tical Character @ecognition 7BC@8

he *rimary 'unction o' BC@ is to allo) searching insi&e the te+t. =ecause )or&s

are o'ten re*eate&, the #" success rate )ill allo) stu&ents an& scholars to 'in&relevant *assage. 6n the *ilot *roects, the BC@ *rogram Abby Fine Reader )as run

http://www.ulib.cs.cmu.edu/






a'ter the scanning )as com*lete&. Abby Fine Reader )as selecte& 'or its ability to

(ee* )or&s intact i' they )ere hy*henate& bet)een t)o *ages. Bn 9nglish language

te+ts )ith *rint that has 'e) bro(en letters, BC@ accuracy o' Abby Fine Reader is

about #" o' te+t. :e &o not *lan to correct the BC@ out*ut as *art o' this

*roect.

More so*histicate& *rograms )ith voting system to resolve &i''erent inter*retations

are available, but licenses are too e+*ensive. Chinese an& Ha*anese BC@ *rograms

are also available an& )ill be use& )henever *ossible. Provi&ing a testbe& that )ill

allo) 'or the creation o' even better BC@ *rograms is a secon&ary goal o' this

*roect. -cholars may )ish to run ne)er BC@ *rograms over the scans an& even to

correct the out*ut.

5. Meta&ata

Digital ibrary e&eration stan&ar&s an& meta&ata best *ractices )ill be use&

throughout this *roect. =ibliogra*hic meta&ata 'or the *ilot *roect )ill be

&erive& 'rom e+isting library catalog recor&s. Carnegie Mellon libraries &evelo*e&

so't)are that uses the stan&ar& F3#.50 *rotocol to search an& retrieve relevant

meta&ata 'rom catalog recor&s 'iel&s. hus, author, title, an& *ublication &ata &o

not have to be re(eye&.

nother research *roect associate& )ith this *roect )ill be the creation o'

so't)are that automatically creates J&ocument structureJ meta&ata. his meta&ataallo)s users to navigate through the cha*ters an& other *arts o' a boo(

success'ully. 9ntering such in'ormation manually is too time consuming 'or this

*roect, but automatic meta&ata creation *rograms can be utilie& subse/uently.

&ministrative meta&ata su**orts the maintenance an& archiving o' the *a*er or

&igital obects an& ensures their longterm availability by *rovi&ing in'ormation

about ho) the 'iles )ere create& an& store&. &ministrative meta&ata )ill be

maintaine& internally as 'ile &escri*tions in the *roect &atabases an& e+ternally as

*art o' the co*yright *ermission &atabase.

he Digital ibrary e&eration, a su**orter o' this *roect, has several initiatives

un&er)ay that )ill allo) commercial bro)sers to harvest meta&ata more

aggressively. he results o' D4s meta&ata harvesting *roect )ill be e+*lore&

'or *ossible a**lication to the resources *ro&uce& in this *roect 7))).&iglib.org8.

!. Kuality control

he stan&ar&s establishe& 'or /uality control are those currently en&orse& by the

Digital ibrary e&eration, )hose missions inclu&e the establishment o' best *ractices an& the &evelo*ment o' stan&ar&s. he *roect must maintain a #"



accuracy rate 'or the /uality o' images an& the inclusion o' all

*ages. evertheless, a *rocess must be &evelo*e& to allo) 'or users to re*ort

missing *ages an& 'or those missing *ages to be scanne& an& &ro**e& bac( into the

e+isting scanne& te+t. =ecause the o)ning library )ill have to *ull the boo(, scan

the *ages, an& trans*ort the 'ile, this *rocess )ill be e+*ensive. Maintaining high/uality the 'irst time the boo( is scanne& )ill be essential. &emonstration o'

high /uality, reliable )or( &one on materials currently in China an& 6n&ia )ill give

U.-. libraries con'i&ence that their collections shoul& be share&.

9. Content

-ee(ing to &evelo* a collection o' one million &igital boo(s, the Million =oo( Proect

envisages a stage& a**roach as &escribe& belo). he Million =oo( *roect )ill a&here

to co*yright la). U.-. collections )ill *rimarily inclu&e the 'ollo)ing ty*es o'

materials.

1. Coor&ination o' -election

Creating one &igital co*y, )hich can then be easily mirrore& in &i''erent locations,

)ill su''ice an& )ill su**ort the multi*le uses an item may receive. Preliminary

&iscussions )ith BCC as a host 'or a registry o' scanne& items are

un&er)ay. Certain (ey *roects, such as the Ma(ing o' merica *roect, are alrea&y

re*resente& in the BCC &atabase as &igital boo(s. Bther large &igitiation *roects

may re/uire some &ata entry o' their content in or&er to avoi& &u*lication.

2. onco*yrighte& materials

Materials *ublishe& be'ore 1#20 are in the *ublic &omain an& may be

scanne& 'or this *roect. -everal large aca&emic libraries are consi&ering

shi**ing materials 'rom their &e*ositories o' little use& material to

6n&ia;China. hese materials )ill be scanne& there an& then returne&.

o re&uce the costs o' selection, the *roect )ill *robably &evelo* a

strategy o' selecting (ey to*ics an& then removing large runs o' boo(s

an& ournals 'rom a selecte& &e*ository. aving a reasonable turn

aroun& time )ill be essential to the success o' the *roect. test )ill be

&evise& to un&erstan& the logistics o' shi**ing the materials an& the

im*act o' their absence 'rom the home library.

he 1#0# co*yright la) grante& co*yright 'or 2" years. @ights hol&ers coul& then

rene) the co*yright 'or another 2" years? many *ublishers an& authors &i& not

e+ercise that rene)al o*tion. hus, some materials *ublishe& a'ter 1#22 75! years

*rior to the 1#$" e''ective &ate o' the 1#$! act8 may be out o' co*yright. 6n or&er

to *rovi&e 'or the e''icient chec(ing o' these boo(s4 status, co*yright rene)alrecor&s 'or boo(s 'or these years been scanne& an& ma&e available online at



))).ulib.org. -imilar recor&s 'or other 'ormats, such as serials an& au&iovisual

material, )ill also be ma&e available as a *art o' this resource.

>overnment &ocuments are also in the *ublic &omain an& may be inclu&e& in this

*roect. Many *artici*ating libraries are &e*ositories 'or 'ull runs o' government&ocuments an& coul& su**ly them to the *roect, as coul& the ibrary o'

Congress. he inclusion o' &ocuments )ill allo) 'or more recent material to enter

the *roect legally an& to become available to a broa&er au&ience an& in a more

accessible manner. Many government &ocuments are currently available in &igital

'orm. he creation o' these bac( 'iles )oul& enhance those resources.

he Chinese &elegation is most eager to have technical re*orts an& science an&

technology &issertations as a *art o' this *roect. he *ro&ucing scholar an& the

university have co*yright interests in these 'ormats. >aining university *ermission

might be 'airly straight'or)ar&. goo& 'aith attem*t )oul& also have to be ma&e

to )in the *ermission o' the scholar. hat coul& be a *art o' an e+ternally 'un&e&

co*yright clearance *roect, but no *ilot has been &one to allo) 'or an estimate o'

contact rate an& subse/uent success. 6' some arrangement coul& be ma&e )ith

University Micro'ilms to scan &issertations o' selecte& universities 'rom micro'ilm,

)hich )oul& be chea*er an& easier to trans*ort, such an initiative might satis'y a

strong &esire among all *artici*ants to increase science content.

3. Co*yrighte& materials

he 1##" Co*yright la) grants co*yright to authors 'or their li'etimes *lus $0

years or 'or #5 years. Patent la), by contrast, gives 20 years. .:. Mellon4s

H-B@ *roect &evelo*e& the conce*t o' a moving )all that allo)e& the inclusion

o' materials over 'ive years ol&. Hournal *ublishers generally agree& that the

economic value o' that material )as greatly re&uce& an& grante& *ermission 'or its

inclusion in this most success'ul *roect. similar broa& *ublisher agreement

about the *oint at )hich economic value o' a *rint boo( &eclines is greatly nee&e&

because boo(s o'ten go out o' *rint in t)o or three years an& can then remain in

co*yright but unavailable 'or over #0 years.

Dr. @a @e&&y an& Dr. Peter -hane, Director o' the 6nstitute 'or the -tu&y o'

6n'ormation, echnology an& -ociety recently ha& a conversation )ith a maor

boo( *ublisher to e+*lore the *ossibility o' ta(ing a broa& *ublisher a**roach to

receiving co*yright *ermissions. Certain *ublishers, inclu&ing the ational

ca&emy Press, have ha& the e+*erience that )hen they &igitie& their boo(s, sales

increase& because attention )as 'ocuse& on the material an& the scholars )ere not

yet rea&y to rea& the boo(s online. uthorsL guil&s )ill also be contacte& to see i'

they )oul& be intereste& in grant *ermissions.



hree con&itions seem to be necessary to attract *ublishers to the scanning o' their

out o' *rint but in co*yright titles

I Publisher shoul& receive a ta+ &e&uction 'or contributing the title to this

*roect. he ta+ &e&uction might re'lect revenues *reviously generate& by thetitle.

I :hen a *rint on &eman& 'eature becomes a *art o' this *roect, *ublishers shoul&

collect royalties on boo(s *rinte&.

I 6' a boo( )ere to return to general *o*ularity, as the e''ect o' the movie itanic

ha& on the sales o' out o' *rint titles, the *ublisher shoul& be able to )ith&ra)

the *ermission 'or a 'ee. he *ublisher might be e+*ecte& to reimburse the

*roect 'or the costs o' &igitiing the title an& maintaining it online.

Dr. Michael -hamos, a Director o' Carnegie Mellon4s Universal ibrary *roect

an& an intellectual *ro*erty attorney, recommen&s the 'ollo)ing a**roach to

co*yright clearance. he million boo( *roect )ill ma(e a goo& 'aith e''ort to

clear co*yright on a**ro*riate materials by sen&ing the *ublisher o' recor& a letter

as(ing 'or *ermission. @e*lies )ill be recor&e& in the a&ministrative meta&ata. 6'

the *ublisher has returne& the rights to the author, the author )ill be

contacte&. -ubse/uent co*yright hol&ers )ill be contacte& as nee&e&. 6' the

*ermission letter receives no res*onse, then materials )ill be &igitie& as a *art o'

the *roect. 6' rights hol&ers subse/uently i&enti'y themselves an& re/uest that the

material be remove& 'rom the *roect, that re/uest )ill be com*lie& )ith

imme&iately.

4. =est boo(s a**roach

he *roect )ill see( *ublisher *ermission to scan boo(s 'rom =oo(s 'or College

ibraries 7=C8, one source 'or core aca&emic boo(s in 9nglish. *revious stu&y

&one at Carnegie Mellon University ibraries in&icates that 22 o' *ublishers

grante& *ermission 'or scanning an& mounting on the )eb. he materials in the

stu&y )ere a ran&om sam*le o' Carnegie Mellon libraries4 boo(s an& inclu&e& a

broa& range o' &ates, *ublishers, an& in an& out o' *rint statuses. umerous

&i''iculties 'rom out o' business *ublishers, lac( o' *ublisher recor&s, return o'

co*yright to authors, an& other circumstances )ere i&enti'ie&. -ubse/uently, Carol

ughes, the collections &evelo*ment o''icer 'or Kuestia, corroborate& Carnegie

Mellon4s e+*erience.

BCC o)ns a &atabase o' boo(s 'rom the latest e&ition o' =oo(s 'or College

ibraries. BCC re*resentatives )ill atten& the ovember 15 1! meeting an&

)ill &iscuss using the &atabase to su**ort the *roect. =C contains about 50,000

titles. 22 success rate in clearing co*yright )oul& result in 10,000 o' the best

boo(s 'or college stu&ents being inclu&e& in the *roect. Clearing co*yright islabor intensive an& e+*ensive. =ra&& =urningham4s recent article estimate&



those costs 77Co*yright Permissions8 in Journal of Interlibrary Loan,

ocu!ent eli"ery, and Infor!ation #upply, 112 720008, #51118. he =C

&atabase, ho)ever, )ill allo) 'or sorting by *ublisher so that *ermission re/uests

can contain the names o' several boo(s. /uic( sam*le in&icates that as many as

25,000 *ublishers may be re*resente& there. Des*ite the e+*ense, this commitmentto /uality shoul& be attem*te&. Carnegie Mellon University ibraries )ill see(

*rivate 'oun&ation 'un&ing to un&erta(e this *roect.

Publishers increasingly see that &igital *resentation o' their )or(s can attract

buyers. hey are intereste& in e+*loring )ays in )hich their out o' *rint titles may

be returne& to *ro'itability. Continue& )or( )ith *ublishers through the course o'

this *roect may attract many o' them to it. hat )oul& be most bene'icial in

enriching the content to be ma&e available.

. -ustainability

-ustainability is a longterm issue 'or this *roect? 'urther research )ill be &one on

&evelo*ing economic mo&els to su**ort this maor contribution to

e&ucation. Partial ans)ers to these signi'icant challenges are &iscusse& belo).

hree general alternatives have *otential 'or o''ering a sustainable mo&el 'or this

*roect;the ibrary o' Congress an& similar national libraries, BCC, an& other

commercial concerns. -everal maor *hilanthro*ists have com*uter in&ustry

'ortunes an& might be intereste& in sustaining this *roect.

Library of Congress$ he millionboo( *roect )ill be a *ublic goo& an& as such

must have a suitable re*ository that )ill continue to ma(e it available to the *ublic

at no charge. hat res*onsibility belongs most clearly to the national library in

each country. he ibrary o' Congress shoul& be motivate& to res*on& to this

challenge because the national interest is so clearly serve&. o)ever, the ibrary

o' Congress is not the national library o' the Unite& -tates, although many *eo*le

assume that it is. 6n the BC4s o)n )or&s in its mission statement 7THE

FIRST PRIORITY o' the ibrary o' Congress is to ma(e (no)le&ge an&

creativity available to the Unite& -tates Congress.8 6t is only a lesser goal to

ma(e (no)le&ge available to the *ublic, an& that is )hy )e have to un&erta(e the

millionboo( *roect in the 'irst *lace BC )on4t &o it. BC is also the

guar&ian o' the co*yright o''ice an& is e+tremely nervous about &igitiing anything

to )hich there might be a co*yright claim. aving a net)or( o' national libraries

mirroring the resource aroun& the )orl& )oul& be an a**ro*riate an& &esire&

outcome.

6n a&&ition, last year, Congress a**ro*riate& 100 million &ollars 'or Digital

Preservation, contingent on C4s raising o' E$5 million in matching

resources. he la) allo)s the acce*tance o' gi'ts in (in& as a *art o' the matching'un&ing. Perha*s the best solution to the sustainability issue )oul& be to *le&ge



the millionboo( *roect to C as a *art o' the Digital Preservation initiative. 9ven

i' the value o' the *roect )ere only assesse& on its in*uts 7e/ui*ment an& labor8, it

&oes re*resent a signi'icant investment. 6nitial overtures have alrea&y been ma&e

'or this alternative.

%CLC$ nother alternative might be 'or BCC to maintain a 'ree version o' the

resource. BCC is a non*ro'it organiation )hose member libraries are

committe& to enhancing access to in'ormation. BCC might cover its costs by

charging member libraries a small 'ee )hen the millionboo( *roect is accesse&

through the 4" milliontitle &atabase. or the millions o' BCC users, that

convenience )oul& be )orth a small *ayment in an alrea&y e+isting 'ee

relationshi*. BCC4s recent strategic *lanning initiatives i&enti'ie& the a&&ition

o' more 'ull te+t to the &atabase, e+*loring archiving res*onsibilities, an&

becoming more international as im*ortant thrusts. BCC )oul& also be able to

cover *artial costs through some o' the strategies liste& belo) 'or *ublishers.

Co!!ercial alternati"es$ he mar(et*lace 'or electronic boo(s is chaotic at this

moment. Kuestia, &esigne& to be an online source )ith at least 50,000 o' the best

boo(s )ith so*histicate& so't)are to su**ort searching an& the creation o'

'ootnotes, mar(ete& itsel' &irectly to stu&ents at a E2030 monthly 'ee. lthough

the *roect )as )ell ca*italie& an& attracte& a great &eal o' me&ia attention, it has

gone out o' business. ibrarians have long observe& that charging 'or resources in

the aca&emic environment re&uces use. -tu&ent &esire 'or the convenience o'

online in'ormation sen&s them to the )eb an& to the muchuse& electronicresources o' their o)n libraries. hat love o' convenience a**arently &oes not

e+ten& to *urchasing Kuestia un&er current *ricing mo&els.

During the same *erio&, the com*any netibrary has announce& that it )ill *rovi&e

o' ne) 'ull te+tboo(s online. etibrary mar(ete& itsel' to libraries through

consortia. Use o' materials, thus, )as at no &irect cost to stu&ents an& 'aculty.

:hile stu&ents a**reciate& the convenience o' being able to use the resource

online, they ha& many com*laints about its 'unctionality;in *articular they

resente& the 'act that boo(s coul& only be *rinte& one *age at a time an& that boo(s

)ere unavailable i' another in&ivi&ual )ere using them. he economic mo&els

behin& netibrary charges also seeme& to re'lect an a&herence to those o' *a*er

boo(s rather than recogniing the economies o' &igital materials. he assets o'

netibrary have since been sol& to BCC.

t this time, the mar(et*lace res*onses suggest that turning the million boo(

*roect into a *rivate, revenuegenerating source )oul& not o''er a sustainable

mo&el. H-B@, Proect Muse, an& other &igital ournal *roects that o''er online

materials )ith su*erior 'unctionality an& sustainability continue to 'lourish. t

some 'uture time, an enhance& version o' the resource might be mar(ete&



commercially i' it o''ere& su''icient a&&e& 'unctionality to encourage a user to *ay

'or using the commercial version rather than the 'ree one.

nother commercial alternative might revolve aroun& relationshi*s )ith

*ublishers. s *ublishers 'in& that ma(ing the boo( available increases sales, theymight be re/uire& to contribute to the su**ort o' the *roect by subscribing to

su**ort buy buttons an& by *aying a *art o' their revenues 'rom *rint on &eman&

sales o' out o' *rint materials.

>. ogistical Challenges

Many logistical challenges 'ace the *roect 18 through*ut on each

scanner, 28 time to com*letion, an& 38 movement o' boo(s to an& 'rom

6n&ia an& China.

%pti!u! scanner t&roug&put$

Bne Minolta scanner running t)o shi'ts &aily N 1!

boo(s *er &ay

250 )or( &ays *er

year 4000 boo(s;year

:ith currently su**lie& 1"

scanners N $2,000 boo(s;year

:ith a total o' 100 scanners100 scanners % 250 &ays;year at 1! boo(s *er

&ay N 400,000 boo(s;year

llo)ing a generous 50 &eterioration in through*ut, 100 scanners can com*lete

the *roect in 'ive years.

China an& 6n&ia have &emonstrate& time an& again that they are the best o' the

&estinations 'or s(ille& man*o)er base& tas(s both 'rom an economic an&

technical com*le+ity *oint o' vie)s.

'i!e to co!pletion$ Decisions about ho) many shi'ts are running;one to three,

ho) many &ays are )or(e& annually, an& ho) aggressively the o*erators are able

to maintain the one boo( *er hour sche&ule. evertheless, the e/ui*ment *rovi&e&

by - an& in&ustry shoul& allo) the *roect to be com*lete& )ithin 'ive years.

Movement of books from the U.S. to India and China: 6n&ian an& Chinese

aca&emic libraries are large an& )ill contain some o' the material to be inclu&e& in

this *roect? over $00 o' these libraries are BCC members )hose hol&ings can beeasily ascertaine&. =ecause the scanner centers )ill be &istribute& aroun& the



countries, they can easily be establishe& in *laces )here the trans*ort o' materials

'rom libraries to them )ill be a minimal &i''iculty. 6t may be 'easible to even

locate some o' the centers in aca&emic libraries.

he tas( o' shi**ing materials 'rom the U.-. to 6n&ia )ill be a monumentalone. he current *lan is to use air containers rather than shi**ing containers to

re&uce the time that materials )oul& be a)ay 'rom their o)ning libraries.

20 + 20 air containers are shi**e& by )eight. 9stimate& cost )oul& be about

E$,500 *er tri*. -uch a container )ill hol& about 2023,000 volumes, )hich can

be *ac(e& at originating libraries in small ven&or bo+es. 6t may be &esirable to

shrin( )ra* the boo(s to ensure their intact arrival. he basic cost 'or shi**ing *er

boo(s )ill be about seventy'ive cents *er roun& tri*. -ome libraries may select

some materials that &o not nee& to be returne& because they are in the *rocess o'

being &iscar&e&, but the &e'ault )ill be that materials )ill return to their

originating libraries. -ome a&&itional 'un&ing is being sought 'rom other sources

to cover library costs in selecting, *ac(ing, an& trans*orting the boo(s. he Center

'or @esearch ibraries might su**ly some o' its o)n materials to the *roect. 6t

might also serve as a collection *lace to)ar&s the en& o' the *roect )hen the

/uantity o' material to be shi**e& 'alls belo) the )hole container level.

. Peo*le at Carnegie Mellon University

he &irectors o' the e+isting Universal ibrary Proect )ill serve as the mainconsultants 'or the *roect. hey are

I Dr. @a @e&&y, Princi*al 6nvestigator, erbert . -imon Pro'essor o' Com*uter

-cience, cochair o' the Presi&ent4s 6n'ormation echnology &visory

Committee 7P6C8, hol&er o' numerous a)ar&s an& *ries.

I Dr. Michael -hamos, Distinguishe& Career Pro'essor in com*uter science an&

intellectual *ro*erty attorney, &irector o' the Universal ibrary *roect, an&

co&irector o' the ecommerce *rogram.

I Dr. Haime Carbonell, llen e)ell Pro'essor o' Com*uter -cience an& Director

o' the anguage echnologies 6nstitute, )hose research interests inclu&e

machine translation, intelligent in&e+ing, autosummariation, an& in'ormation

mining. his *roect )ill *rovi&e the in'rastructure nee&e& 'or many o' the

research activities o' Carbonell an& associates in 6.

I Dr. @obert hiba&eau, Princi*al research scientist )ith areas o' s*ecialiation in

scanning e/ui*ment an& in areas o' *rivacy

I Dr. >loriana -t. Clair, Princi*al 6nvestigator, University ibrarian, an& e&itor

o' portal$ Libraries and t&e Acade!y an& an active member o' the Digital

ibrary e&eration.

&&itional university libraries *ersonnel inclu&e



I Ms. >abrielle Michale(, Digital library *roects coor&inator, success'ul lea&er o'

the scanning o' 1 million *ages o' &igital archival materials, the 100 boo(

*roect an& the 1000 boo( *roect.

I Ms. 9ri(a in(e, ssociate University ibrarian, )ith e+*ertise in collectionmanagement, &igital libraries, an& intellectual *ro*erty issues.

I Ms. Denise roll, ssociate University ibrarian an& Distinguishe& ello) o' the

Digital ibrary e&eration, )ith broa& com*etencies in &igital libraries an&

s*ecial interests in user stu&ies.

Peo*le else)here

I Dr. Chingchih Chen, Pro'essor o' the >ra&uate -chool o' ibrary an&

6n'ormation -ciences, -immons College, member o' P6C

I Dr. >ao :en, Pro'essor o' Com*uter -cience, <ice Presi&ent o' University o'

-cience an& echnology o' China, De*uty Presi&ent o' the >ra&uate -chool o'

Chinese ca&emy o' -ciences.

I Dr. . =ala(rishnan, hea& o' the 6n&ian 6nstitute o' -ciences Division o'

6n'ormation -ciences

I Dr. Daniel >reenstein, 9+ecutive Director o' the Digital ibrary e&eration.

Appendix: Chinese Collections

C&inese collections -everal o' the Chinese Universities *artici*ating in the *roect have i&enti'ie&collections they )ill )ant to scan initially. China uses a &i''erent system 'or intellectual

*ro*erty. **ro*riate *ermissions )ill be secure& to scan the materials inclu&e& in this

*roect. -i+ other universities may also contribute materials.

=eiing University

I ncient rare boo(s inclu&ing -ong an& Auan Dynasty rare boo(s, 'amily trees,

*aintings, an& inscri*tion rubbings.

I Chinese *erio&icals be'ore 1#4# in *olitics, la), culture, e&ucation, 'inance,

economics, stu&ents, )omen, aca&emics, technology, religion, 'ol( customs, an& natural

sciences.

Tsingh%a 3niversityI ncient engineering technology history an& stu&y in China

I China4s contribution to -cience an& echnology, inclu&ing engrave& bone te+ts.

<%dan 3niversityI ull te+t o' &ocuments in the Chinese Culture Documents Database

I ull te+t o' &ocuments in the Chinese Classical iterature Database

I ull te+t o' &ocuments in the Chinese Classical art vision &atabase



=anjing 3niversityI ull te+t o' &ocuments in the Hinling University echnical Perio&ical &atabase

I 7-i+&ynasties Culture8 multime&ia &atabase in coo*eration )ith the library

aning normal school

I ai*ing eavenly Oing&om materials

>hejiang 3niversityI Dunhuang &ocuments, inclu&ing han& )ritten an& carve& ones.

I China4s -outheast countrysi&e area )ith local area geogra*hy, commercial to)n

materials, an& history an& technical articles.

I ea culture materials, ancient &ocuments, *erio&icals, te+ts

I -il()or( an& sil( materials, inclu&ing ancient &ocuments, technical articles, te+ts

**en&i+ ? 6n&ian Collections

he 6n&ian agencies *artici*ating in the *roect have &one e+tensive stu&ies an& i&enti'ie& the

&ocuments that are *recious, uni/ue to the regions an& are beyon& all co*yright issues. Close to

about 1000 &ocuments an& boo(s have alrea&y been &igitie& at the three o*erational centers.

I 6n&ian 6nstitute o' -cience

I 6t is one o' the ol&est an& the largest - ibraries )ith more than 400,000 hol&ings

an& out o' this collection nearly 40,000 are estimate& to be out o' co*yright.

I 6nternational 6nstitute 'or 6n'ormation echnology, y&eraba& an& the >overnment o'

n&hra Pra&esh, y&eraba&

I elugu e+t boo(sI 6n&ian 6nstitute o' 6n'ormation echnology

I -ans(rit iterature an& - =oo(s in 9nglish an& 6n&ian anguages available 'rom

=ose ibrary, llahaba&.

I Pune University

I Maharastrian iterature an& boo(s

I >oa University

I Portughese iterature an& =oo(s

I iru*athi an& irumala Devasthanam

I -ans(rit an& elugu iterature an& ve&ic &ocuments, *alm leaves

I nna University

I amil iterature, Palm leaves containing ancient 6n&ian me&ical *ractices

7yurve&a8

I ational Centre 'or -o't)are &evelo*ment an& the >overnment o' Maharastra

I e+t =oo(s in Marati an& - =oo(s

I --@

I -ans(rit an& amil iterature 'rom anore library &ating bac( to 4th Century =C

I vinashalingam College

I =oo(s an& manuscri*ts 'rom ol& libraries in the amilna&u region in amil, elugu,

9nglish an& -ans(rit.