Hurley -- 12/3/98 1

The Making of America IIThe Making of America II

ProjectProject

http://sunsite.berkeley.edu/moa2http://sunsite.berkeley.edu/moa2

Bernie HurleyBernie Hurley

Chief Scientist, UC Berkeley LibraryChief Scientist, UC Berkeley Library

(bernie@library.berkeley.edu)(bernie@library.berkeley.edu)

Hurley -- 12/3/98 2

Today’s AgendaToday’s Agenda

I. MoA II Project BackgroundI. MoA II Project Background

II. Planning Phase -- MoA II White PaperII. Planning Phase -- MoA II White Paper

III. Production Phase -- Progress to DateIII. Production Phase -- Progress to Date

Hurley -- 12/3/98 3

I. MoA II Project BackgroundI. MoA II Project Background

• Standards Setting ProjectStandards Setting Project– Continuation of the Encoded Archival Description Continuation of the Encoded Archival Description

(EAD) Project(EAD) Project• Electronic finding aids for archival materialsElectronic finding aids for archival materials

– MoA II Goal is to Create Community Standards MoA II Goal is to Create Community Standards for Digital Library Objectsfor Digital Library Objects

• Including metadata and contentIncluding metadata and content

• Builds on previous work in the communityBuilds on previous work in the community

• Help understand what value metadata adds to scholarly Help understand what value metadata adds to scholarly and instructional use of the objects -- and at what costand instructional use of the objects -- and at what cost

Hurley -- 12/3/98 4

MoA II Project Background (Cont.)MoA II Project Background (Cont.)

• Three PhasesThree Phases– The Planning Phase The Planning Phase (DLF funded)(DLF funded)

• Write and review a Write and review a MoA II White PaperMoA II White Paper to begin the to begin the standards discussion and prepare for the testbedstandards discussion and prepare for the testbed

– The Production Phase The Production Phase (NEH Funded)(NEH Funded)

• Create a testbed that will allow the community to Create a testbed that will allow the community to experiment with and evaluate different practices for experiment with and evaluate different practices for digital archival object discovery, display, navigation and digital archival object discovery, display, navigation and manipulationmanipulation

– Dissemination PhaseDissemination Phase

Hurley -- 12/3/98 5

II. The MoA II White PaperII. The MoA II White Paper

Hurley -- 12/3/98 6

II. MoA II White Paper OrganizationII. MoA II White Paper Organization

• Part 1: Executive Summary and Project Part 1: Executive Summary and Project

BackgroundBackground

• Part 2: The MoA II ModelPart 2: The MoA II Model

• Part 3: Implementing the MoA II ModelPart 3: Implementing the MoA II Model

• Part 4: Best Practices for ImagingPart 4: Best Practices for Imaging

• AppendicesAppendices– Bibliography, Sample DTD and object encodings, Bibliography, Sample DTD and object encodings,

etc. etc.

Hurley -- 12/3/98 7

Part 1: White Paper Executive SummaryPart 1: White Paper Executive Summary

Goals of the White Paper1) Start a discussion on creating “community standards” 1) Start a discussion on creating “community standards”

for digital library objectsfor digital library objects• Review and discussion of this paper is the 1st stepReview and discussion of this paper is the 1st step

2) Create a MoA II “holistic” model to frame the 2) Create a MoA II “holistic” model to frame the discussiondiscussion

• Why we need these standards? What do they include? Why we need these standards? What do they include? Where do they fit into a larger National DL model?Where do they fit into a larger National DL model?

3) Start an investigation of best practices for the 3) Start an investigation of best practices for the digitization of imagesdigitization of images

4) Gather information needed for a MoA II testbed4) Gather information needed for a MoA II testbed

Hurley -- 12/3/98 8

Part 2: The MoA II ModelPart 2: The MoA II Model

The goal of the MoA II Model is to is to develop suites develop suites of tools for specific audiences that integrate of tools for specific audiences that integrate the discovery, display, navigation and the discovery, display, navigation and manipulation of standardized objects across manipulation of standardized objects across distributed repositories.distributed repositories.

Hurley -- 12/3/98 9

The MoA II Model: AssumptionsThe MoA II Model: Assumptions

1) A National DL will be made up of 1) A National DL will be made up of many many different classesdifferent classes of objects of objects– library, archival, museum, GIS, numeric datasetslibrary, archival, museum, GIS, numeric datasets

2) These objects will populate 2) These objects will populate distributed distributed repositoriesrepositories

3) Scholars and students will require 3) Scholars and students will require coherent coherent and integrated accessand integrated access to these objects (i.e., to these objects (i.e., distributed repositories are transparent)distributed repositories are transparent)

Hurley -- 12/3/98 10

Assumptions (Cont.)Assumptions (Cont.)

4) Therefore, we will need to develop 4) Therefore, we will need to develop toolstools that that can transparently discover, display, navigate can transparently discover, display, navigate and manipulate DL objects across distributed and manipulate DL objects across distributed repositoriesrepositories

5) 5) These tools will require objects be standardizedThese tools will require objects be standardized– Note: Having standardized objects may also be Note: Having standardized objects may also be

helpful to libraries in sharing tool development helpful to libraries in sharing tool development costs and in the archiving of these objectscosts and in the archiving of these objects

Hurley -- 12/3/98 11

MoA II Model and Sub-modelsMoA II Model and Sub-models

The MoA II Model:The MoA II Model: Goal:Goal: Suites of tools for specific audiences that Suites of tools for specific audiences that

integrate the discovery, display, navigation and integrate the discovery, display, navigation and manipulation of standardized objects across distributed manipulation of standardized objects across distributed repositories.repositories.

– Digital Library Service Model

• Digital Object Model

– Distributed Systems Architecture Model• Not yet a funded part of the MoA II ProjectNot yet a funded part of the MoA II Project

Hurley -- 12/3/98 12

MoA II -- Digital Library Service ModelMoA II -- Digital Library Service Model

Services

Tools

Digital Objects

(Digital Object Sub-model)

Hurley -- 12/3/98 13

Service Model (cont.)Service Model (cont.)

1) Services1) Services – Suites of Tools that support a particular audience that support a particular audience– E.g. K-12 vs. scholarly use of a photograph E.g. K-12 vs. scholarly use of a photograph

collectioncollection

2) Tools2) Tools– Client/Server applications that consist of Client/Server applications that consist of suites of

user level behaviors that allow for the discovery, that allow for the discovery, display, navigation and manipulation of display, navigation and manipulation of standardized digital objectsstandardized digital objects

Hurley -- 12/3/98 14

Service Model (cont.)Service Model (cont.)

– User level behaviorsUser level behaviors (functionality) (functionality)• Behaviors are what tools do for usersBehaviors are what tools do for users

• Diary Example: How are you organized (dated entries, Diary Example: How are you organized (dated entries, accounts, contacts)?; get first/next/previous/last entry accounts, contacts)?; get first/next/previous/last entry

3) Digital Library Objects3) Digital Library Objects– Foundation of the Service ModelFoundation of the Service Model– Need to be standardized so tools can work on them Need to be standardized so tools can work on them

across distributed repositories across distributed repositories– Encapsulate content and metadataEncapsulate content and metadata

Hurley -- 12/3/98 18

Review: Digital Library Service ModelReview: Digital Library Service Model

Services

Tools

Digital Objects

Content; Metadata; Methods

The goal of the MoA II Model is to develop suites of tools for specific audiences that integrate the discovery, display, navigation and manipulation of standardized objects across distributed repositories.

Hurley -- 12/3/98 19

MoA II Architecture ModelMoA II Architecture Model

-Class Tools

-Objects

Repository

-Class Tools

-Objects

Repository

-Class Tools

-Objects

Repository

Workstation

Network

Union Index

Client/Server,Client/Server,Object OrientedObject Oriented

Hurley -- 12/3/98 20

Java

Applet(tool)

Web Server

Applet

WebBrowser

ORB

Workstation

InternetObject

Repository

An ObjectAn ObjectWEB WEB

ImplementationImplementationObjectRepository

INTERNET

Hurley -- 12/3/98 21

MoA II Applet/CORBA Implementation MoA II Applet/CORBA Implementation

DBMS(Content)

WebServer

CORBA Repository Layer(server side behaviors)

UnionCatalog

Create HTML page w/display

applet

HTTP CORBA

1: Object ID2: Object ID3: Object ID

Display Applet?Repository & Object ID

1) Browser: Discovery Applet

ORB

2) Browser: Display Applet

Repository

ORB

Hurley -- 12/3/98 22

Part 3: Implementing the MoA II ModelPart 3: Implementing the MoA II Model• Select Classes for the MoA II TestbedSelect Classes for the MoA II Testbed

– Continuos tone photographs, Photo albums, Diaries, Continuos tone photographs, Photo albums, Diaries, Correspondence, Journals, LedgersCorrespondence, Journals, Ledgers

• MoA II Digital Library Service ModelMoA II Digital Library Service Model1) Services1) Services

• Audience is scholars Audience is scholars

2) Tools2) Tools• Discover, display and navigate objects in selected classesDiscover, display and navigate objects in selected classes

• BehaviorsBehaviors– General NavigationGeneral Navigation– Images: Pan , Zoom, Color comparison, etc.,Images: Pan , Zoom, Color comparison, etc.,

Hurley -- 12/3/98 23

MoA II Model Implementation (cont.)MoA II Model Implementation (cont.)

3) Digital Library Objects (Model)3) Digital Library Objects (Model)a) Content a) Content

– Images, text, encoded textImages, text, encoded text

b) Metadatab) Metadata

– Descriptive: MARC collection records & finding aidsDescriptive: MARC collection records & finding aids

– Structural & Administrative elements defined in detailStructural & Administrative elements defined in detail

c) Methods c) Methods – To be determined by system designerTo be determined by system designer

• Create a DTD to Encode (encapsulate) Create a DTD to Encode (encapsulate) Metadata and Content Metadata and Content – Transfer SyntaxTransfer Syntax

Hurley -- 12/3/98 24

Part 4: Best Practices for ImagingPart 4: Best Practices for Imaging

• General RecommendationsGeneral Recommendations– Scanning, Digital masters & derivatives, Image Scanning, Digital masters & derivatives, Image

quality, File formatsquality, File formats

• Specific Project RecommendationsSpecific Project Recommendations– 24 bit color for color images24 bit color for color images– 8 bit grayscale for non-color images8 bit grayscale for non-color images– Most images should be captured at 600 DPI or Most images should be captured at 600 DPI or

greatergreater– Avoid compressionAvoid compression– Use TIFF, but record which versionUse TIFF, but record which version

Hurley -- 12/3/98 25

III. Production Phase III. Production Phase Progress to DateProgress to Date

Hurley -- 12/3/98 26

MoA II --Progress to DateMoA II --Progress to Date

1)1) White Paper White Paper

2)2) The MoA II DTD The MoA II DTD

3)3) Digitization Management Software Digitization Management Software – MS Access DatabaseMS Access Database– MoA II Object BuilderMoA II Object Builder

4)4) Tools: A MoA II Compliant Viewer Tools: A MoA II Compliant Viewer

Hurley -- 12/3/98 27

1) MoA II White Paper1) MoA II White Paper

• Reviewed by the MoA II ParticipantsReviewed by the MoA II Participants

• Submitted to the Digital Library Federation Submitted to the Digital Library Federation for publicationfor publication

• Found at http://sunsite.berkeley.edu/moa2Found at http://sunsite.berkeley.edu/moa2– Use the Postscript version if possibleUse the Postscript version if possible

Hurley -- 12/3/98 28

2) The MoA II DTD2) The MoA II DTD

• Version 1.0 is CompleteVersion 1.0 is Complete

• The DTD, a Tutorial Explaining the DTD and The DTD, a Tutorial Explaining the DTD and an Example Encoded Document can be found an Example Encoded Document can be found at http://sunsite.berkeley.edu/moa2/at http://sunsite.berkeley.edu/moa2/

• DTD has 3 SectionsDTD has 3 Sections– File InventoryFile Inventory– Administrative Metadata SectionAdministrative Metadata Section– Structural MapStructural Map

Hurley -- 12/3/98 29

2a) File Inventory2a) File Inventory

<FileGrp VERSDATE='1998-09-01' ADMID='HIREZJPG <FileGrp VERSDATE='1998-09-01' ADMID='HIREZJPG IPRIGHTS IMGSRC'> IPRIGHTS IMGSRC'>

<File ID='HRJ1' MIMETYPE='image/jpg' SEQUENCE='1' <File ID='HRJ1' MIMETYPE='image/jpg' SEQUENCE='1' SIZE='220346' CREATED='1998-09-01' SIZE='220346' CREATED='1998-09-01'

OWNERID='CPR-hi.jpg'> OWNERID='CPR-hi.jpg'>

<FLocat LOCTYPE='URL'> <FLocat LOCTYPE='URL'> http://sunsite.berkeley.edu/~jmcdonou/CPR-hi.jpg</FLocat> http://sunsite.berkeley.edu/~jmcdonou/CPR-hi.jpg</FLocat>

</File> </File>

</FileGrp> </FileGrp>

Hurley -- 12/3/98 30

2b) Administrative Metadata2b) Administrative Metadata

<AdminMD ID="HIREZJPG"> <AdminMD ID="HIREZJPG">

<FileMgmt> <FileMgmt>

<Image> <Image>

<Compression>JPEG</Compression> <Compression>JPEG</Compression>

<Dimensions X='1536' Y='1024' /> <Dimensions X='1536' Y='1024' />

<BitDepth BITS='16' /> <BitDepth BITS='16' />

<ColorSpace>RGB</ColorSpace> <ColorSpace>RGB</ColorSpace>

<CLUT FResident='YES'></CLUT> <CLUT FResident='YES'></CLUT>

<Resolution>200 DPI</Resolution> <Resolution>200 DPI</Resolution>

</Image> </Image>

</FileMgmt> </FileMgmt>

Hurley -- 12/3/98 31

2c) Structural Map2c) Structural Map

<StructMap> <StructMap>

<div N='1' TYPE='Stereograph' LABEL='Secrettown. 62 <div N='1' TYPE='Stereograph' LABEL='Secrettown. 62 miles -- altitude 3,000 feet. miles -- altitude 3,000 feet.

Photographer&#39;s number: 47'> Photographer&#39;s number: 47'>

<fptr FILEID='HRJ1' MIMETYPE='image/jpeg' /> <fptr FILEID='HRJ1' MIMETYPE='image/jpeg' />

<fptr FILEID='LRJ1' MIMETYPE='image/jpeg' /> <fptr FILEID='LRJ1' MIMETYPE='image/jpeg' />

<fptr FILEID='GIF' MIMETYPE='image/gif' /> <fptr FILEID='GIF' MIMETYPE='image/gif' />

</div> </div>

</StructMap> </StructMap>

Hurley -- 12/3/98 32

3) Digitization Management Software3) Digitization Management Software

• Too Difficult to Create MoA II Objects by Too Difficult to Create MoA II Objects by HandHand

• Realized That Most Metadata can be Captured Realized That Most Metadata can be Captured During the Digitization ProcessDuring the Digitization Process

• Create Digitization Management SoftwareCreate Digitization Management Software– The MoA II DatabaseThe MoA II Database

• Object definition and data captureObject definition and data capture

• Built on MS Access databaseBuilt on MS Access database

– MoA II Object BuilderMoA II Object Builder

Hurley -- 12/3/98 33

3a) MoA II Database3a) MoA II Database

• Part 1:Part 1: Object DefinitionObject Definition– Define objects and sub-objects to digitizeDefine objects and sub-objects to digitize– Defining the structureDefining the structure

• Part 2:Part 2: Image ManagementImage Management– Define parameters for master and derivativesDefine parameters for master and derivatives– Capturing administrative metadataCapturing administrative metadata– Generates file names for each imageGenerates file names for each image

• Part 3:Part 3: DefaultsDefaults

Hurley -- 12/3/98 34

3b) MoA II Object Builder3b) MoA II Object Builder

• Program that runs through the MoA II Program that runs through the MoA II database and creates fully encoded MoA II database and creates fully encoded MoA II objectsobjects

Hurley -- 12/3/98 35

4) Tools -- MoA II Compliant Viewer4) Tools -- MoA II Compliant Viewer

• Java AppletJava Applet

• Used RMI (Remote Method Invocation) to Used RMI (Remote Method Invocation) to Connect to a Distributed RepositoryConnect to a Distributed Repository

• Series of Method Calls to Retrieve parts of the Series of Method Calls to Retrieve parts of the ObjectObject

Hurley -- 12/3/98 36

Hurley -- 12/3/98 37

Hurley -- 12/3/98 38

Hurley -- 12/3/98 39

The END!The END!