escidoc persistence layer architecture. page overview 1.general escidoc architecture 2.business...

eSciDoc Persistence LayerArchitecture

Overview

1. General eSciDoc architecture

2. Business Layer Excursion

3. Persistence Layer

4. Transformation to the module/dependency structure

Overview

1. General eSciDoc architecture

Web Application

Spring Framework

PersistenceLayer

Business LogicLayer

CommunicationLayer

TransportLayer

CXF

Secu

rity

Inte

rcep

tor

Serv

ice

Spec

.

REST

En

dpoi

ntSp

ec.

JMS

Tran

spor

t

JAXB

REST

En

dpoi

ntIm

pl.

SOAP

End

poin

tSp

ec.

SOAP

End

poin

tIm

pl.

HTT

PTr

ansp

ort

Fast

Info

Set I

nter

cept

or

Serv

ice

Impl

.

XO

P In

terc

epto

r

GZI

P In

terc

epto

r

Repo

sito

ry S

pec. FE

DO

RA

Repo

sito

ry

Impl

.

Tran

sacti

on In

terc

epto

r

JSON

H

iber

nate

FEDORA

RMDB

implements

depends

TransferObjects

DomainObjects

PersistentObjects

mapmap

RMD

BRe

posi

tory

Im

pl.

1.1 Focus

Web Application

Spring Framework

PersistenceLayer

Business LogicLayer

CommunicationLayer

Serv

ice

Spec

.

Endp

oint

Spec

.

Endp

oint

Impl

.

Serv

ice

Impl

.

Repo

sito

ry S

pec.

Repo

sito

ry Im

pl.

DataMaintenance

System

implements

depends

TransferObjects

DomainObjects

PersistentObjects

CL-M

appi

ng

PL-M

appi

ng

1.2 Mapping

CL- & PL-Mappings are part of each layer Only the layer knows, which peaces of the business data belongs to the objects of the

layer Example: Hibernate

The Hibernate Layer defines the data structure to be used to persist the data (tables, indexes, etc.)

Hibernate Model A: Use one table to store Items and Container

Hibernate Model B: Use two different tables to store Items in one table and Containers in the other table

Example: Fedora Fedora uses FOXML to persist the data

Example: Communication Layer (CXF) Defines Transport-Objects which are marshalled to XML, JSON etc.

Because of the different data structures each concrete layer instance implements, it is

the layers responsibility, to map the business domain objects to its own data model The business logic is completely independent of anything that happens outside of its

own borders

2. Business Layer

A little excursion for later reference

Business Logic LayerService Spec.

Service Impl.

updateItem()

updateMdRecords()

updateMdRecord()

updateContainer()

Domain Objects

Item ContainerMdRecordsMdRecord …

Persistence Spec.

Uses Domain Object

Possible Interface/Call

Internal call to sub-method

2. Implementation structure

Structured, centralized and reusable code Sub-resource methods reused by the service-methods handling the main resources Avoid conceptual flaws on implementation of new features, bug fixes and so on

(protect the developer by design)

Service methods implement the eSciDoc logic like Update a resource:

What happens, if the resource from the client equals the currently persisted resource

and what happens if not?

Service methods use the Domain Objects as data holders, which only

implement logic, that belongs to their semantic like getObjid() : extracts the objid from the xlink:href getVersionNumber() : extracts the version number from getObjid()

Thanks to this concept, sub-resource methods can be reused and offered as

services to the communication layer as well

2. Flow example

updateItem(arg0)

updateMdRecords(arg0, arg1)

updateMdRecord(arg0, arg1)

updateMdRecords(mdRecords, null)

updateItem(item, null)

updateMdRecord(mdRecord, null)

arg1 == null ? arg1 = retrieveMdRecords()

arg1 = retrieveItem()

arg0.hasMdRecords() & arg1.hasMdRecords()

updateMdRecord(arg0.getMdRecord(name), arg1.getMdRecord(name))

arg1 == null ? arg1 = retrieveMdRecord()arg0 != arg1 ? persistMdRecord()

CL PL

updateMdRecords(arg0.getMdRecords(),arg1.getMdRecords())

arg0.hasMdRecord(name) & arg1.hasMdRecord(name)

2. Loss of information

Providing central methods for reusability will cause a loss of information eSciDoc interfaces are structured to support different resource types per interface The centrally provided methods however do not differ between these resources, when

handling sub-resources The persistence layer may still need the information about the type of the resource

If the Hibernate Model B is being used Or if the data is stored in different ways into the FOXML, depending on the

resource type Therefore we have to support the ResourceType for these methods, not matter if we

need the type or not Support future expandability and plug-in ability for the persistence layer

Signature example: updateMdRecord(MdRecord, MdRecord, ResourceType) Support an extra package/module for the static sub-resource handlers for better

understanding of the structure

3. Persistence Layer

Goals Current goal for SCAPE: Use Hadoop as a persistence layer Plug-in ability

In theory eSciDoc could get released without an already implemented persistence

layer How the customer can plug-in an own implementation of the persistence layer is

not part of this presentation (How to do this in Tomcat, JBoss etc.) Black box for the business layer

Exception handling Mapping of the business domain objects to its own objects

3. Persistence Layer as a Black Box

Exception handling Exceptions thrown in the persistence layer cannot be handled in detail by the

business layer The business layer does not know, what kind of persistence layer is being used Plug-in support offers customers to implement their own persistence layer, we

do not even know anything about their implementations The business layer may define an abstract PersistenceException, which will be the

representative of all exceptions thrown in the persistence layer If we have some code in the current business layer, handling such persistence

layer exceptions, we have to analyze this code and find a better solution for the

idea of this code ResourceNotFoundException

Return null instead of throwing an exception in the persistence layer and throw a

ResourceNotFoundException in the business layer because it is the business

logic decision and design to throw an exception, if a resource was not found

Persistence LayerBusiness LayerCommunication Layer

3. Exception handling – Flow example (ResourceNotFoundException)

retrieveItem()

Retrieve ItemRetrieve Item Retrieve Item

Load data

Throw 404 exception

return exception XML

Return Itemreturn resource

XML Resource found

Delegate Persistence Exception

Error-like exceptionIO)

404 exceptionOR empty resultOR anythingalike

Return null

throw PersistenceException

Return resource

3. Exception handling - Outcome

Avoid conceptual flaws on implementation Non persistence layer exceptions should not be available for the persistence layer Adjust eSciDoc module hierarchy Adjust the persistence layer interfaces to describe the expected behavior of the

business layer Implement the mapping between business domain objects and persistence layer

objects as already described

3. Plug-in ability

Now we can add any kind of persistence layer implementation to the

business layer by making them configurable in SpringBusiness Logic

Layer

Serv

ice

Spec

.

Serv

ice

Impl

.

implements

depends

DomainObjects

Hibernate Model A

Repo

sito

ry

Spec

.

Repo

sito

ry

Impl

.

DataMaintenance

System

PersistentObjects

PL-M

appi

ng

Hibernate Model B

Repo

sito

ry

Spec

.

Repo

sito

ry

Impl

.Data

MaintenanceSystem

PersistentObjects

PL-M

appi

ng

Fedora

Repo

sito

ry

Spec

.

Repo

sito

ry

Impl

.

DataMaintenance

System

PersistentObjects

PL-M

appi

ng

…

3. Plug-in ability

But wait…

There is something very wrong with this architecture!

3. Plug-in ability

Each Persistence layer defines its own specification.

Business LogicLayer

Serv

ice

Spec

.

Serv

ice

Impl

.

implements

depends

DomainObjects

Hibernate Model A

Repo

sito

ry

Spec

.

Repo

sito

ry

Impl

.

DataMaintenance

System

PersistentObjects

PL-M

appi

ng

Hibernate Model B

Repo

sito

ry

Spec

.

Repo

sito

ry

Impl

.Data

MaintenanceSystem

PersistentObjects

PL-M

appi

ng

Fedora

Repo

sito

ry

Spec

.

Repo

sito

ry

Impl

.

DataMaintenance

System

PersistentObjects

PL-M

appi

ng

…

3. Persistence Specification

If the specification is part of the persistence layer, the business logic has to implement

a switch in order to select and work with the current persistence layer The business logic has to implement special code, which deals with each persistence

layer specification The business logic layer depends on the persistence layer The business logic layer cannot specifiy the expected behavior of the persistence

layer (for example: the exception handling) The persistence layer specification expects the persistent objects in its signatures and

because of that the business logic has to perform the mapping from its domain objects

to the persistent objects, no matter what kind of persistence layer we are using The persistence layer is no real black box A lot of ugly code will be written

All this conflicts with the idea to support the plug-in ability and to achieve an

easy extension to use Hadoop for example

3. Corrected architecture

Web Application

Spring Framework

PersistenceLayer

Business LogicLayer

CommunicationLayer

Serv

ice

Spec

.

Endp

oint

Spec

.

Endp

oint

Impl

.

Serv

ice

Impl

.

Repo

sito

ry Im

pl.

DataMaintenance

System

implements

depends

TransferObjects

DomainObjects

PersistentObjects

CL-M

appi

ng

PL-M

appi

ng

Repo

sito

ry S

pec.

3. Outcome

The business logic layer defines the persistence specification Uses its domain objects in the signatures Predefines the expected behavior (Exception handling etc.) Uses the PersistenceException in the signatures

Each persistence implementation implements this one specification and performs its

own mapping from domain objects to its own persistent objects The persistence layer becomes a real black box The business logic layer can easily work with any persistence layer implementation

without the need to implement special code for any persistence layer implementation The persistence layer is pluggable like the communication layer

4. eSciDoc Modules> Communication Layer

context-rest-impl

context-rest-spec

dataobjects-impl

dataobjects-spec

jax-rs framework

om

policies

aspects

common

aa

utilities

spring aspects

FastInfoSet

geronimo-servlet 3.0 spec

woodstox-core-asl CXFfrontend-jaxrs

Transports (HTTP, Jetty, JMS)

Business logic spec.

Communication Layer

Business Logic Layer

BL dataobjects (domain objects)

Mapping


context-rest-impl

context-rest-spec

dataobjects-impl

dataobjects-spec

jax-rs framework

om

policies

aspects

common

aa

utilities

spring aspects

FastInfoSet




Before we continue, we have to clean this up.

Mapping


context-rest-impl

context-rest-spec

dataobjects-impl

dataobjects-spec

jax-rs framework

om

policies

aspects

common

aa

utilities

spring aspects

FastInfoSet




?

Dependencies are resolved transitively. „aspects“ & „policies“ should be included in each root module only Business logic modules cannot be used in the communication layer and vise versa. Only the business logic specification should be used in module „context-rest-impl“

Mapping

4. eSciDoc Modules> Communication Layer (Discussion)

dataobjects-impl depends on aa Reason unknown

Dependencies utilities and common Reusability

Not allowed between layers Create a utility module outside of the layers

Dependency to om Reason unknown

4. eSciDoc Modules> Communication Layer (Cleanup)

context-rest-impl

context-rest-spec

dataobjects-impl

dataobjects-spec

jax-rs framework

om

policies

aspects

common

aa

utilities

spring aspects

FastInfoSet




?


Communication Layer


Mapping


4. eSciDoc Modules> Communication Layer (Final)

context-rest-impl

context-rest-spec

dataobjects-impl

dataobjects-spec

jax-rs framework

policies

aspectsspring aspects

FastInfoSet





Communication Layer


Mapping


dataobjects

4. eSciDoc Modules> Communication Layer (Final)

context-rest-impl

context-rest-spec

dataobjects-impl

dataobjects-spec

jax-rs framework

policies

aspectsspring aspects

FastInfoSet





See architecture on page 19

Use woodstox as the parser impl. for JAXB

General servlet spec.(Also works for JBoss)

?

No need to differ between spec. & impl. for objects

Communication Layer

Business Logic Layercall

implement

Mapping


international standard that specifies a binary encoding format for the XML Information Set (for JMS?)

4. eSciDoc Modules> Business logic layer


Business logic Impl.

common

policies

aspects

Persistence layer spec.

aa

Sub-resource handler

See page 19

See page 7 - 14

Optional(Should be replaced with an HTTP connection or using CXF in terms of clustering abilities later)

Other dependencies (no eSciDoc modules)


implementcall

4. eSciDoc Modules> Business logic layer (Discussion)

dataobjects-spec is required for SRU classes for example but should not be part of this

layer (RequestType, ResponseType, etc.) Create dataobjects (domain objects) module for business layer, where the domain

objects are being implemented and external objects are generated from XSD

schema files Code about mapping between String & DateTime should not occur in this layer

(DateTimeJaxbConverter from the Communication Layer) Put such utility classes in an utility module, which can be reused on any layer and

is not part of any layer Rename DateTimeJaxbConverter to DateTimeConverter (for example)

4. eSciDoc Modules> Persistence layer

Persistence layer Impl.

policies

aspects

Persistence layer spec.

Other dependencies (no eSciDoc modules)

PL dataobjects


Mapping

implement


Persistent Layer

4. eSciDoc Modules> Persistence layer (Discussion)

Nothing

4. Project Structure & Dependencies

eSciDoc BL Project

eSciDoc CL CXF Project eSciDoc PL Fedora Project

Domain Objects

BL Spec

PL Spec

BL Impl.

Utilities

4. Project Structure & Dependencies

Advantages The borders of the layers are clear Protects the developer to spuriously add wrong dependencies

Maybe add some kind of Maven plug-in, which evaluates the allowed eSciDoc

module dependencies If a new persistence or communication layer is being implemented, a new project

for this new layer can be created Avoids adding a bunch of new modules to the current project for the new layer

implementation(s) The Utilities module has to be completely independent from any layer

If that is not possible, because you may need objects from one layer, it has to be

part of this layer and cannot be reused in another layer!

Addendum A - Layers

Why do the CL, BL and PL layers looks so differently? Let us look at this using another perspective and some examples

BL

BL Spec PL Spec

CL CXF REST

CL Axis SOAP

GUI PL Fedora PL Hibernate

PL Hadoop

OutgoingIncoming

call implement

“Plug-in”

Requ

est

Request

Addendum B – BL Project Structure

Protect the developer by design Reusability of some modules for internal implementations of CL and PL

<POM>

BL

<POM>

Tools

<POM>

Core

<POM>

API

<JAR>

JAX-RS Framework

<JAR>

Utilities

<JAR>

AA

<JAR>

OM

<JAR>

Domain Objects

<POM>

PL API

<POM>

BL API

<JAR>

AA PL API

<JAR>

OM PL API

<JAR>

AA BL API

<JAR>

OM BL API

…

…

…

Addendum B – BL Project StructureArtifact Group-IDs

<POM>

BL

<POM>

Tools

<POM>

Core

<POM>

API <JAR>

Domain Objects

<POM>

PL API

<POM>

BL API

org.escidoc.bll.tools

org.escidoc.bll.core

org.escidoc.bll.api.domain

org.escidoc.bll.api.persistence

org.escidoc.bll.api.business


Tools Offer our JAX-RS Framework as a “service” to be reused by the CL & PL

implementations as we require it in our current implementations Like the RestEASY service from JBoss

Offer some general and independent Utilities API

Domain Objects Required by the BL API and PL API and therefore have to be available to the

public as well PL API

Separated into each eSciDoc module (aa, om, oum etc.) BL API

Separated into each eSciDoc module (aa, om, oum etc.) Core

Separated into each eSciDoc module (aa, om, oum etc.)


Pros of this structure You can see the borders within the BL and therefore know, which modules can

depend on each other and where dependencies are forbidden No dependencies are allowed across Tools, Core and API Tools and API are the only module groups, which can be referenced by other

projects/layers (such as PL, CL and the BL itself) No layer (PL or CL) is allowed to depend on modules of the Core module group

Implement a Maven plug-in, which evaluates the dependencies and causes the

build process to fail, if a dependency to the Core module group exists Simply do not allow dependencies to org.escidoc.bll.core* Use this plug-in in CL & PL projects and in API & Tools module group too

Addendum B – BL Project StructureReusability

Our internal implementations of CL and PL use the JAX-RS Framework module as

their CXF implementation Avoid redefinition of the same module in both layers BL may use this framework too

<POM>

BL

<POM>

Tools

<JAR>

JAX-RS Framework

<JAR>

Utilities PLCL

Addendum C – Example for Context

JAX-RS Server

context-rest-spec

context-rest-impl

context-spec

context-impl

context-pl-api

context-pl-impl

FedoraRestClient

FedoraRestClientImpl

FedoraRESTEndpoint

JAX-RS Client

CL

BL

BL

PL

TODO

External (not a module)

Addendum C – Example for AA

JAX-RS Server

aa-rest-spec

aa-rest-impl

aa-spec

aa-impl

aa-pl-api

aa-pl-impl

Hibernate

Hibernate Impl

CL

BL

BL

PL

TODO

External (not a module)

Thank you !

These documents are intended for presentation purposes only. Copyright lies with FIZ Karlsruhe.Any distribution or use of these documents or part thereof issubject to FIZ Karlsruhe's express approval.

© FIZ Karlsruhe 2011

escidoc persistence layer architecture. page overview 1.general escidoc architecture 2.business...

Documents