ICAT Integration at ISIS
Tom Griffin, ISIS FacilityICAT Developer WorkshopThe Cosener’s House, AbingdonAugust [email protected]
ISIS• Neutrons and Muons• Operating since 1984• 1600 user base; ~200 staff• Volume of data ~6TB• ~5M datafiles (27 instruments,
500/hour)• >6.7GB metadata, >33M rows
Current Pre-Experiment
Data.ISIS
Data.ISIS• Main point of access for ISIS data• Maintains index of Run number -> Folder
– GEM12345– \\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw– .txt .log .sav0
• Ranges• Zip files
Data.ISIS Authentication
Data Portal
ICAT API
Data.ISIS
User performs search via application e.g. Data PortalSearch is executed in ICATPermitted results are returned to application
1
1
2
3
3
2
Results are displayed to the user4
4
5
User performs request to download datafile, multiple datafiles or dataset
5
6
ICAT creates http GET link and passes to back to user (routed through application)
sessionIdemail (optional)fileId(s) or datasetIdaction (i.e. download, zip,
compressed)
6
User clicks http link7
7
Data.ISIS call ICAT API to check permissionssessionId & datafileId(s) or
datasetId
8
8
Return Exception on failure or DownloadObject on success- userId- array [filename, cycle, run
number]
9
9
User gets their data!10
10
Current Pre-Experiment
‘To Be’ Pre-Experiment
Problems• RB Numbers• Business System
• Not released to users• Replace DataPortal with TopCat
Related Projects• Grid MP – distributed PC network• Analysis and simulation• New portal required anyway ->
1) Single point of access for Grid MP2) Use ISIS data in Grid jobs3) Store results and process in ICATWhere to store data?
Freddie Akeroyd (Data.ISIS)Chris Moreton-Smith (ISIS DFS/FRS)Hannah Guest (Facilities Business System & ERAS)Matt Clarke (SECI modifications)
Damian FlanneryStuart Ansell (writeRaw)
Laurent Lerusse (NXIngest)
Database Services, Roger Downing, FBU IT Service Desk (servers and hosting)
Core ICAT – Glen, Shoaib, Michael.....
Acknowledgments(who did what at ISIS)
ISIS Computing Group
ISIS Data Analysis Group
Questions...