#CMIMI18#CMIMI18

Towards a Scalable Infrastructure for Medical Deep

Research

Peter D Chang MD, and Anna Alber, MSCenter for Artificial Intelligence in Diagnostic Medicine

University of California Irvine

#CMIMI18#CMIMI18

How to Scale?

dataset curation algorithm development deploymentexperiment design

anonymizationfile format conversion

database storageannotation

cohort identificationground-truth

DICOM download

image preprocessingnetwork design

hyperparameter tuningvalidation

DICOM networkingGUI interface

communication

1 2 3 4

Other:distributed file server

GPU clustershigh-speed LAN

IRB approvalReusable components: >90%Custom components: <10%

#CMIMI18

Machine-learning algorithm across the UC Irvine Healthcare system

Current support for 25 data scientists, software developers, research faculty and affiliated students working on over a dozen independent, high capacity medical deep learning projects.

Scalable design to handle numerous simultaneous users each annotating, writing code and training algorithms across numerous GPU clusters

#CMIMI18

KEY REQUIREMENTS

Transfer and anonymization directly from the PACS

Scalable system for distributed annotation across many individual users

Storage of raw high resolution 2D and 3D imaging data and DICOM metadata with high I/O speed

#CMIMI18

KEY REQUIREMENTS

Expandable system for file storage with availability across all GPU nodes

Scalable, easily-accessible (e.g. non-technical) system for developing code on remote GPU clusters

#CMIMI18

OVERVIEW OF DEEP LEARNING STRUCTURE

#CMIMI18

HARDWARE

File storage: 0.24 PB configured in RAID-5 array served via 10 GbEhigh-speed LAN GPUs: 20 GPUs (combination of Titan X and GTX 1080 Ti cards) in x4

clusters

#CMIMI18

DATA IMPORT

Pacs networking

Dcm4che(SCP/SCU)

Mirc (anonymization)

Memory-mapped NumPy for imaging data storage

MongoDD ( DICOM metadata)

Docker containers for portable systems

#CMIMI18

CODE DEVELOPMENT ENVIRONMENT

JupyterLab notebooks served using multi-user JupyterHub system

Reusable deep learning library facilitating rapid prototyping of networks

Docker containers for portable deployment

#CMIMI18

CODE DEVELOPMENT

#CMIMI18

CODE DEVELOPMENT

Python 3.5 Tensorflow 1.5 Java JavaScriptNodeJSWebGL, Cornerstone CUDA 9.0 Custom Medical DL Libraries

#CMIMI18

WEB-BASED ANNOTATION AND VIEWING PLATFORM

Secure web-based annotation platform designed using Cornerstone and WebGL

User interface to interact with imaging data

Monitoring algorithm training over time and eventual algorithm deployment Pixel-level, bounding box, global data

annotation

#CMIMI18

SUMMARY

The goal is to provide research community with state of the art infrastructure, methods and models to accelerate research in AI.

Ensure low entry threshold in medical image analysis and deep learning coding.

Provide reusable, easy to maintain platform for multiple users involved in numerous deep learning projects