plugin lifecycle andrew fabian metaarchive annual membership meeting atlanta, georgia friday october...
TRANSCRIPT
Plugin Lifecycle
Andrew FabianMetaArchive Annual Membership
MeetingAtlanta, Georgia
Friday October 24, 2008
Plugin Lifecycle
START
Local Development and Testing
Commit toRepository
New CollectionsIn Conspectus
IsolationTesting
Plug-inModifications
ConspectusModifications
Production! END
Local Development
Note: There is a presentation on this topic by Daniel Culpepper.
• Write the Plugin with LOCKSS Plugin Tool.
• Test-crawl with that tool.– Crawling will happen from your desktop computer,
not your LOCKSS box. Make sure firewalls are OK with this.
• This implies setting up a Manifest page on the site to be crawled.
• When finished you have an .XML Plug-in file.
Subversion
• Subversion synchronizes files between MetaArchive developers.
• You’ll need a Subversion client.– TortoiseSVN for Windows– SCPlugin for Macs– svn (command-line) for everything
• After a plugin is written, add it to the MetaArchive repository.
• Releases (testing and production) are made from the official repository only.
The Conspectus Database• This is the MetaArchive’s online master list of
what gets harvested.
• It contains:– Title– Descriptive Metadata– Base URL– Plugin Identifier– Extra Parameters
• Add a new record for the collection you’re getting ready to harvest.– These are fairly detailed records. Make sure to have the
appropriate descriptive metadata on hand.
Isolation Testing
• This tests two things:– The final combination of the Plugin, Collection,
and production software– Recrawls
• MetaArchive Sandbox:– VMWare Virtual Machine– Gets Plugins from Subversion, Collections from
the Conspectus– Runs the MetaArchive LOCKSS daemon, but
does not communicate with the rest of the network
Going Live
• Almost the same process as Isolation Testing
• Ask MetaArchive admins to release the plugin and the collection on a weekly phone call.
• They will install the plugin for the entire network, and allocate space on a set of MetaArchive storage nodes to cache the new collection.
Plugin ModificationsIn Production
• Go through normal development cycle
• Make sure to increment the Plugin Version Number!
• Ask admins to update the Plugin from Subversion– All MetaArchive caches will load the new plugin within a day.– For faster updates, trigger a manual crawl of the “Plugins”
AU from the DebugPanel.
Conspectus ModificationsIn Production• Edit Descriptive Metadata at any time
• Add additional AUs at any time– Add a new line to the “Extra Parameters”
section– Announce the new AU via email and weekly
phone call
• Changing existing AU parameters can break things. Instead, add a new AU and then delete the old.
Deletion
• If a collection no longer needs to be harvested, and the backup copy is no longer useful, it can be deleted.
• Deletion is tricky, because there are several steps:– Delete Collection from Conspectus– Delete AUs from all participating MetaArchive Caches– Delete files from disk of all participating MetaArchive
caches
• Detailed instructions are in the MetaWiki.