Sentinel ToolSentinel Tool

June 11, 2009

2

Sentinel ToolSentinel Tool

• Overview• Architecture• Implementation• Dependencies• Futures

3

Sentinel Tool - OverviewSentinel Tool - Overview

• Watches caDSR activity and content• Curate Alert Definitions• Report on activity to subscribers per Alert

Definitions• Perform independent audit for caDSR content• Perform QA for metadata• Notify Sentinel Administrator of activity• Support all members of the community, e.g. form

designers, modelers, curators, etc• Provide metadata history, when available• Automatically notify software systems of activity

via XML messages

4

Sentinel Tool - OverviewSentinel Tool - Overview

• Stakeholders:– Form Designers– Modelers– Developers– Analysts– Clinicians– Statisticians– Researchers– Curators– caBIG– NCI

5

Sentinel Tool - ArchitectureSentinel Tool - Architecture

• Technologies– Java 1.5– Javascript– HTML 4– JDBC– Struts– EVS 4.2

6

Sentinel Tool - ArchitectureSentinel Tool - Architecture

• Struts / JSP / HTMLView

• JBossController

• Java 1.5Application

• Class, InterfaceModel

• JDBC, PL/SQL, ANSI SQLDatabase

• Oracle 10gPersist

7

Sentinel Tool - ArchitectureSentinel Tool - Architecture

• Auto-deploy– Deployable via Anthill– Ant –DPROP.FILE=… build-all deploy

• SCM– CVS– .cvsignore for all transient files– One file, no duplicates, e.g. template.web.xml vs. web.xml

• All files placed in deployment-artifacts• Production deployment artifacts

– Accessible via links in email from Anthill– Files hosted on GForge for distribution– URL references to GForge hosting for Wiki, Download, etc

8

Sentinel Tool - ArchitectureSentinel Tool - Architecture

• Jboss/cadsrsentinel.war– Web Browser UI– Curation for Alert Definitions

• Gforge/cadsrsentinel.jar– API interface for Monitor, Unmonitor, etc

• Bin/autorun.sh– Deploys to /local/content/cadsrsentinel/bin/.– Automated nightly jobs– Purges report cache– Scrubs caDSR of control characters, e.g. [^[:space:]]– Generates and broadcasts Audit Report and Alert Reports– Refreshes materialized views– Scheduled and launched by CRON

9

Sentinel Tool - ArchitectureSentinel Tool - Architecture

tool name

• CURATION• SENTINEL• …

property• URL• EMAIL• …

value• https://cdecurate...•sentinel@scenpro.com• …

…• …• …• …

Tool Options Table

• Tool options table hosts configuration values beyond 3rd party requirements, e.g. XML

• Dynamic– Values are read as needed – user sees changes in real time– Values cached when new session created – user must close

window– Values never cached with application – requires restart of

JBoss

10

Sentinel Tool - ArchitectureSentinel Tool - Architecture

• SQL script updates/sets tool option values– Updates limited to SENTINEL tool name

• SQL may check database schema during deployment

– E.g. When a new column is added to a table/view a SELECT using the column name will throw an error if the database is not updated before deploying the tool

• SQL may *never* alter schema• SQL may perform data migration• Must be coordinated and negotiated with caDSR

database deployment scripts• Login performs tool options verification

11

Sentinel Tool - ImplementationSentinel Tool - Implementation

• Project Structure– Conf

Configuration files, e.g. XML, which require value substitution during build and deployment

– Db-sqlSQL scripts which are run during build and deployment or as part of a manual process and for database refreshes

– DocPatterned after phases in development lifecycle with the addition of “Administration” for all documentation specific to NCI policies and processes and not directly pertinent to the product features

• Administration• Construction• Elaboration• Inception• Transition

– LibJAR files needed for building the project *but* not included in the deployment, e.g. ojdbc14.jar is deployment on Jboss and not packaged in project WAR but must be present to compile and build the WAR, allows for the separation of the build machine and the deployment target machine

12

Sentinel Tool - ImplementationSentinel Tool - Implementation

• Project Structure– Scripts

Console scripts to manage temporary file system space, create Alert Reports, refresh materialized views, etc.

– SrcJava source, more details follow

– WebRootThe deployed cadsrsentinel.war content

• Css• Html• Images• Js• Jsp• Meta-inf• Web-inf

– Lib– Tld

13

Sentinel Tool - ImplementationSentinel Tool - Implementation

• Packages– gov.nih.nci.cadsr.sentinel.audits

Creates audit reports– gov.nih.nci.cadsr.sentinel.daily

Scrub control characters and refresh materialized views– gov.nih.nci.cadsr.sentinel.database

caDSR database interface and implementation– gov.nih.nci.cadsr.sentinel.tags

Tag support for JSP– gov.nih.nci.cadsr.sentinel.test

StrutsUnit automated tests– gov.nih.nci.cadsr.sentinel.tool

Main business logic– gov.nih.nci.cadsr.sentinel.ui

Web Browser UI using Struts– gov.nih.nci.cadsr.sentinel.utl

Utility features, e.g. Monitor, Unmonitor, etc

14

Sentinel Tool - ImplementationSentinel Tool - Implementation

• Logging– Cadsrsentinel_log.txt

Jboss messages from gov.nih.nci.cadsr.sentinel.*– Server.log

Jboss messages from 3rd party packages, e.g. struts– Report.txt

Messages from the control character scrub and materialized view refresh

– Autorun_log.txtMessages from the Alert Report and Audit Report generation

– /local/content/cadsrsentinel/reportsLog files and report output for Alert Definitions

15

Sentinel Tool - DependenciesSentinel Tool - Dependencies

• Audit Report for caDSR/EVS concepts– Curation Tool

• Specific Vocabularies defined in tool options under the CURATION tool name

– EVS• Currently uses the 4.2 remote client API• Adopter can remove Audit Report from

template.cadsrsentinel.xml

• Oracle 10g– The regexp_* functions are used in several of places (these

are for regular expression comparisons and replacement)

16

Sentinel Tool - FuturesSentinel Tool - Futures

• Upgrade to EVS 5.0• Additional Audits for CS/CSI circular references,

etc• Update Edit page to improve performance• Update List page with new UI paradigm from

Curation Tool• Add Public Id and Version to

https://cadsrsentinel.nci.nih.gov/cadsrsentinel/do/loghtml?version=1&idseq=...

• Improve performance for “loghtml”• Move manual run outside Jboss to reduce resource

requirements• Improve performance for Alert Report generation