using little rails apps to solve a hefty problem pairing durable content with lightweight apps matt...
Post on 26-Mar-2015
212 Views
Preview:
TRANSCRIPT
QuickTime™ and aNone decompressor
are needed to see this picture.
Using Little Rails Apps to Solve a Hefty Problem
pairing durable content with lightweight apps
Matt ZumwaltHydraCamp
05 October 2010
QuickTime™ and aNone decompressor
are needed to see this picture.
“the web” vs. deep web
Think about all of the digital stuff that humanity is creating right now...
QuickTime™ and aNone decompressor
are needed to see this picture.
“the web” vs. deep web
QuickTime™ and aNone decompressor
are needed to see this picture.
“the web” vs. deep web
How deep does it run?
How deep could it run?
QuickTime™ and aNone decompressor
are needed to see this picture.
“the web” vs. deep web
QuickTime™ and aNone decompressor
are needed to see this picture.
“the web” vs. deep web
QuickTime™ and aNone decompressor
are needed to see this picture.
what are we doing?
Imagine capturing all of humanity’s digital output from a single day.
How long would it take to go through it all?
How much could we learn about 2010 from that corpus?
Is any of it worthless? What’s worth preserving?
QuickTime™ and aNone decompressor
are needed to see this picture.
what could we do?What deep information would be
useful if it was online in a searchable, machine-parseable structure?
census data? crime data?
climate data? genomics?
government documents?
medical research?
QuickTime™ and aNone decompressor
are needed to see this picture.
curation
creating, using and preserving
embrace connections
embrace re-use
many perspectives
QuickTime™ and aNone decompressor
are needed to see this picture.
curate what?how much deep information is
currently online in useful formats?
how much of the knowledge we produce is being preserved?
what will your answer be in 10 years?
QuickTime™ and aNone decompressor
are needed to see this picture.
golden age of digital information
wide open fields of possibilities
ownership, copyright & channels of distribution have not been frozen yet
QuickTime™ and aNone decompressor
are needed to see this picture.
long lived, complex contentthe data we create extend beyond and outlast the software used to
create them“the open data movement is more
important than the open source movement”
- Ian Davis Code4Lib 2009
QuickTime™ and aNone decompressor
are needed to see this picture.
the funny problemOur vision is to create complex, evolving, long-lived content. Meanwhile, each set of users wants to see & interact with that content in different ways relative to temporal and intellectual context.
content in open, complex, long-lived context
users in task-specific, time-
specific context
QuickTime™ and aNone decompressor
are needed to see this picture.
content in open context, users in specific context
QuickTime™ and aNone decompressor
are needed to see this picture.
QuickTime™ and aNone decompressor
are needed to see this picture.
let’s take a look at some of the apps we’ve written
QuickTime™ and aNone decompressor
are needed to see this picture.
key points• allow applications to be small and specialized
• build iteratively, allowing app to evolve as you learn & have ideas
• reuse your good code, find patterns
• use your model as skeleton then customize views
• make testing & prototyping easy
• think carefully about generators & pre-fab plugins
QuickTime™ and aNone decompressor
are needed to see this picture.
standing astride two different
worlds
SOA
Content Modeling
QuickTime™ and aNone decompressor
are needed to see this picture.
conflicting sensibilities & requirements
• flash-in-the-pan applications vs. long lived content
• agile development vs. big design up front
• the importance of developer happiness
• convention over configuration vs. standards & interoperability
• "XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
• "Enterprise is a dirty word" vs. "Enterprise is our reality"
QuickTime™ and aNone decompressor
are needed to see this picture.
flash-in-the-pan applications vs. long lived content
allow apps to be short lived while content is long lived
agile development vs. big design up front
keep your eyes on the horizon (big goals) and your feet on the ground (iterative development)
the importance of developer happiness
start talking about developer happiness & get developers talking to each other
convention over configuration vs. standards & interoperability
put standards in their place. when they serve you, make them the assumed (thus hidden) convention
"XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
XML is a serialization, NOT a programming language
"Enterprise is a dirty word" vs. "Enterprise is our reality"
use the strengths of enterprise to neutralize its pitfalls
QuickTime™ and aNone decompressor
are needed to see this picture.
many lightweight views onto
complex content
QuickTime™ and aNone decompressor
are needed to see this picture.
flash-in-the-pan applications vs. long lived content
allow apps to be short lived while content is long lived
agile development vs. big design up front
keep your eyes on the horizon (big goals) and your feet on the ground (iterative development)
the importance of developer happiness
start talking about developer happiness & get developers talking to each other
convention over configuration vs. standards & interoperability
put standards in their place. when they serve you, make them the assumed (thus hidden) convention
"XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
XML is a serialization, NOT a programming language
"Enterprise is a dirty word" vs. "Enterprise is our reality"
use the strengths of enterprise to neutralize its pitfalls
QuickTime™ and aNone decompressor
are needed to see this picture.
eyes on the horizon,
feet on the groundThe iterative approach of agile development, combined with RAD, transforms the dynamic
between an application’s developers, its users, and the users’ tasks.
Detachment from reality is the real
danger.
Big designs, lofty goals, and detailed descriptions of needs are all great.
(Don’t build the death star.)
QuickTime™ and aNone decompressor
are needed to see this picture.
flash-in-the-pan applications vs. long lived content
allow apps to be short lived while content is long lived
agile development vs. big design up front
keep your eyes on the horizon (big goals) and your feet on the ground (iterative development)
the importance of developer happiness
start talking about developer happiness & get developers talking to each other
convention over configuration vs. standards & interoperability
put standards in their place. when they serve you, make them the assumed (thus hidden) convention
"XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
XML is a serialization, NOT a programming language
"Enterprise is a dirty word" vs. "Enterprise is our reality"
use the strengths of enterprise to neutralize its pitfalls
QuickTime™ and aNone decompressor
are needed to see this picture.
developer happinessHappy developers write
better, more innovative software.
QuickTime™ and aNone decompressor
are needed to see this picture.
flash-in-the-pan applications vs. long lived content
allow apps to be short lived while content is long lived
agile development vs. big design up front
keep your eyes on the horizon (big goals) and your feet on the ground (iterative development)
the importance of developer happiness
start talking about developer happiness & get developers talking to each other
convention over configuration vs. standards & interoperability
put standards in their place. when they serve you, make them the assumed (thus hidden) convention
"XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
XML is a serialization, NOT a programming language
"Enterprise is a dirty word" vs. "Enterprise is our reality"
use the strengths of enterprise to neutralize its pitfalls
QuickTime™ and aNone decompressor
are needed to see this picture.
put standards in their place
When they serve you, make standards the assumed, thus
hidden convention. When they’re harmful, say so.
Convention over configuration is not at odds with standards, it merely gets to the heart of
the matter.
QuickTime™ and aNone decompressor
are needed to see this picture.
flash-in-the-pan applications vs. long lived content
allow apps to be short lived while content is long lived
agile development vs. big design up front
keep your eyes on the horizon (big goals) and your feet on the ground (iterative development)
the importance of developer happiness
start talking about developer happiness & get developers talking to each other
convention over configuration vs. standards & interoperability
put standards in their place. when they serve you, make them the assumed (thus hidden) convention
"XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
XML is a serialization, NOT a programming language
"Enterprise is a dirty word" vs. "Enterprise is our reality"
use the strengths of enterprise to neutralize its pitfalls
QuickTime™ and aNone decompressor
are needed to see this picture.
XML is a serialization
%table#search_results
%th Title
%th Narrator
%th Location
%th Subject Headings
%tr.result
%td My Title
%td Nancy Narrator
%td Boston, Ma
%td Subject Heading -- Detail, Subject Heading -- Detail II
QuickTime™ and aNone decompressor
are needed to see this picture.
XML is a serialization<table id='search_results'>
<th>Title</th>
<th>Narrator</th>
<th>Location</th>
<th>Subject Headings</th>
<tr class=”result”>
<td>My Title</td>
<td>Nancy Narrator</td>
<td>Boston, Ma</td>
<td>Subject Heading -- Detail, Subject Heading -- Detail II</td>
</tr>
</table>
QuickTime™ and aNone decompressor
are needed to see this picture.
XML is a serialization
• HTML model has become more important than the serialization
• JSON is very useful
• XSLT is declarative (good for speed, bad for dynamic code) and XML-centric (ivory tower)
• Example: ORE & RDF are both models first while serializations are interchangeable
QuickTime™ and aNone decompressor
are needed to see this picture.
flash-in-the-pan applications vs. long lived content
allow apps to be short lived while content is long lived
agile development vs. big design up front
keep your eyes on the horizon (big goals) and your feet on the ground (iterative development)
the importance of developer happiness
start talking about developer happiness & get developers talking to each other
convention over configuration vs. standards & interoperability
put standards in their place. when they serve you, make them the assumed (thus hidden) convention
"XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
XML is a serialization, NOT a programming language
"Enterprise is a dirty word" vs. "Enterprise is our reality"
use the strengths of enterprise to neutralize its pitfalls
QuickTime™ and aNone decompressor
are needed to see this picture.
teach the beast to dance
Use the strengths of enterprise to your benefit while side-stepping its pitfalls.
QuickTime™ and aNone decompressor
are needed to see this picture.
flash-in-the-pan applications vs. long lived content
allow apps to be short lived while content is long lived
agile development vs. big design up front
keep your eyes on the horizon (big goals) and your feet on the ground (iterative development)
the importance of developer happiness
start talking about developer happiness & get developers talking to each other
convention over configuration vs. standards & interoperability
put standards in their place. when they serve you, make them the assumed (thus hidden) convention
"XML is a quagmire" vs. XML assumed as preservation format, validation point, a basis of interoperability, and (sometimes) an integral part of users' vernacular
XML is a serialization, NOT a programming language
"Enterprise is a dirty word" vs. "Enterprise is our reality"
use the strengths of enterprise to neutralize its pitfalls
QuickTime™ and aNone decompressor
are needed to see this picture.
storing files & metadata
Database-Driven Fedora Repositoryie. Paperclip
demo:5demo:5xyz.mp3xyz.mp3
DC metaDC metatech metatech metarelationshirelationshi
psps......
/uploads/xyz.mp3/uploads/tuv.doc/uploads/blf.jpg
.../xyz.mp3
.../demo-5.foxml
tuv.doctuv.doc.../tuv.doc
QuickTime™ and aNone decompressor
are needed to see this picture.
CRUD
CreateRetrieveUpdateDelete
QuickTime™ and aNone decompressor
are needed to see this picture.
CRUD
Create/SubmitModifyDeleteIndex
QuickTime™ and aNone decompressor
are needed to see this picture.
CRUD
SearchBrowse
View DetailsRender Edit View
QuickTime™ and aNone decompressor
are needed to see this picture.
CRUD in Rails
Rails MVCRails MVC
RDBMSRDBMS
QuickTime™ and aNone decompressor
are needed to see this picture.
CRUD in repositories
Create/SubmitCreate/Submit(CUD)(CUD)
Search/ViewSearch/View(R)(R)
IndexingIndexing
QuickTime™ and aNone decompressor
are needed to see this picture.
the Hydra framework
Hydra Rails Hydra Rails EngineEngine(CUD)(CUD)
BlacklightBlacklight(R)(R)
SolrizerSolrizerFedoraFedora SolrSolr
http://github.com/projecthydra/hydrangeahttps://wiki.duraspace.org/display/hydra/
QuickTime™ and aNone decompressor
are needed to see this picture.
rich, faceted search
BlacklightBlacklight(Read only)(Read only)
SolrSolr
• faceted search & discovery• metadata agnostic• content-aware• supports specialized users• Rails plugin• solr-driven • strong dev community
http://projectblacklight.org
QuickTime™ and aNone decompressor
are needed to see this picture.
models that work
ActiveFedoraActiveFedora
FedoraFedora
• models inspired by datamapper• xml tools -- OM• convenience methods• update_attributes
OMOM
http://github.com/mediashelf/active_fedorahttp://github.com/mediashelf/om
QuickTime™ and aNone decompressor
are needed to see this picture.
rails engine
FedoraFedora
ActiveFedoraActiveFedora
CustomCustomControlleControlle
rsrs
Custom Custom Views & Views & HelpersHelpers
OMOM
• ActiveFedora & OM• controllers integrate everything• views & helpers to simplify dev• custom JQuery plugin
QuickTime™ and aNone decompressor
are needed to see this picture.
the final piece: indexing
Hydra Rails Hydra Rails EngineEngine(CUD)(CUD)
BlacklightBlacklight(R)(R)
??????FedoraFedora SolrSolr
QuickTime™ and aNone decompressor
are needed to see this picture.
the Hydra framework
Hydra Rails Hydra Rails EngineEngine(CUD)(CUD)
BlacklightBlacklight(R)(R)
SolrizerSolrizerFedoraFedora SolrSolr
http://github.com/projecthydra/solrizer
QuickTime™ and aNone decompressor
are needed to see this picture.
now the fun begins...
top related