deep learning algorithms for liver and tumor ......results: mri liver tumor segmentation n 20 test...
TRANSCRIPT
![Page 1: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/1.jpg)
© Fraunhofer
Medical Knowledge Through Research
Grzegorz Chlebus, Hans Meine
DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR SEGMENTATION
![Page 2: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/2.jpg)
© Fraunhofer
Medical Knowledge Through Research
Deep learning is a major thing
Before 2013
n We failed to design image analysis algorithms that perform better than humans.
Feature
ExtractionClassifier
![Page 3: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/3.jpg)
© Fraunhofer
Medical Knowledge Through Research
Deep learning is a major thing
2013 – now
n We found that deep learning works well for image understanding tasksthanks to faster computers and better training algorithms.
Deep neural network
![Page 4: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/4.jpg)
© Fraunhofer
Medical Knowledge Through Research
DL algorithms can surpass expert performance
![Page 5: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/5.jpg)
© Fraunhofer
Medical Knowledge Through Research
DL algorithms can surpass expert performance
![Page 6: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/6.jpg)
© Fraunhofer
Medical Knowledge Through Research
What medical tasks can be solved by DL?
n Detection Is it present or not?
n Classification What type of thing is it?
n Segmentation How big is it, what shape does it have?
n Prediction What are the chances that this patient will get cancer in X years from now?
n Recommendation Which therapy option would be the best for this patient?
![Page 7: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/7.jpg)
© Fraunhofer
Medical Knowledge Through Research
Automatic liver and tumor segmentation
n Automate/improve the planning process of liver interventions
n SIRT planning
n Basis for tumor load computation
n Required for dose computation
n Manual or semi-automatic segmentation
n Tedious and time consuming
n Inter-observer variability
Motivation
![Page 8: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/8.jpg)
© Fraunhofer
Medical Knowledge Through Research
CT Data
n LiTS Challenge dataset
n 131 CT scans with reference segmentations of liver and tumors
n ~0.8 mm in-plane resolution
n ~1.5 mm slice thickness
n Liver surgery planning dataset
n 179 CT scans with reference segmentations of liver
n ~0.6 mm in plane-resolution
n ~0.8 mm slice thickness
![Page 9: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/9.jpg)
© Fraunhofer
Medical Knowledge Through Research
MRI Data
n SIRTOP dataset
n 90 DCE-MRI scans with reference liver and tumor segmentations
n Acquired at Städtisches Klinikum Dresden, Germany
n 0.74-1.76 mm in-plane resolution
n 2-5 mm slice thickness
Native 20s 60s 120s 15 min
![Page 10: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/10.jpg)
© Fraunhofer
Medical Knowledge Through Research
Segmentation Pipeline
n OrthoMean [1]
[1] Prasoon A et al., “Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network”, MICCAI 2013.
![Page 11: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/11.jpg)
© Fraunhofer
Medical Knowledge Through Research
Results: CT Liver Segmentation
n 40 test cases
n Automatic method: 79 points according to MICCAI score [1]
n Trained human performance (no radiological expert): 75 points
[1] Heimann T et al., “Comparison and Evaluation of Methods for Liver Segmentation from CT Datasets”, IEEE TMI 2009.
![Page 12: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/12.jpg)
© Fraunhofer
Medical Knowledge Through Research
Results: CT Liver Tumor Segmentation
n 30 test cases
n Automatic Method
n 0.58 Dice per case
n 0.69 Dice per tumor
n MTRA performance
n 0.7 Dice per case
n 0.72 Dice per tumor
Dice
![Page 13: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/13.jpg)
© Fraunhofer
Medical Knowledge Through Research
Results: CT Liver Tumor Segmentation
n State-of-the-art results
n 3rd place at MICCA round of the LiTS challenge
n 28 teams
![Page 14: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/14.jpg)
© Fraunhofer
Medical Knowledge Through Research
Results: MRI Liver Segmentation
n 28 test cases
n Automatic method: 0.95 Dice [1]
n Human performance: 0.94-0.95 Dice
n 1 radiologist
n 2 residents
[1] Chlebus G et al., “Automatic Liver and Tumor Segmentation in Late-Phase MRI Using Fully Convolutional Neural Networks”, CURAC 2018.
![Page 15: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/15.jpg)
© Fraunhofer
Medical Knowledge Through Research
Results: MRI Liver Segmentation
[1] Chlebus G et al., “Automatic Liver and Tumor Segmentation in Late-Phase MRI Using Fully Convolutional Neural Networks”, CURAC 2018.
n 28 test cases
n Automatic method: 0.95 Dice [1]
n Human performance: 0.94-0.95 Dice
n 1 radiologist
n 2 residents
![Page 16: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/16.jpg)
© Fraunhofer
Medical Knowledge Through Research
Results: Training Data Size
n Liver segmentation quality in MRI
![Page 17: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/17.jpg)
© Fraunhofer
Medical Knowledge Through Research
Inter-observer variability
n Routine vs. corrected liver segmentations
n Ca 35% of slices were corrected (3 observers)
n Average 5 min per case correction time
![Page 18: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/18.jpg)
© Fraunhofer
Medical Knowledge Through Research
Results: MRI Liver Tumor Segmentation
n 20 test cases
n Automatic method: 0.65 Dice [1]
n Human performance: 0.90-0.93 Dice [2]
[1] Chlebus G et al., “Automatic Liver and Tumor Segmentation in Late-Phase MRI Using Fully Convolutional Neural Networks”, CURAC 2018.
[2] Budjan J et al., “Semi-automatic Volumetric Measurement of Treatment Response in Hepatocellular Carcinoma after TACE”, 2016.
![Page 19: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/19.jpg)
© Fraunhofer
Medical Knowledge Through Research
Summary
n Deep learning algorithms are very successful at image analysis tasks
n Deep learning methods can help radiologist to perform their work faster and more accurate
n Liver segmentation quality of our automatic method was comparable to that of human segmentations
n Tumor segmentation is a more difficult task than liver segmentation
n Acquiring more training data has a positive impact on the model performance
n Future work
n More extensive validation
![Page 20: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/20.jpg)
© Fraunhofer
Medical Knowledge Through Research
Thank you for your attention J
Questions?
![Page 21: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/21.jpg)
© Fraunhofer
Medical Knowledge Through Research
Automatic liver and tumor segmentation
Reduce inter-observer variability
n RECIST 1.1 study by Bellomi et al. [1]
n 100 radiologists
n 3 cases
Conclusion
Age and expertise of the radiologist remain the most critical factors.
Motivation
[1] Bellomi M. et al. “Evaluation of inter-observer variability according to RECIST 1.1 and its influence on response classification in CT measurement of liver metastases” 2017.
![Page 22: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/22.jpg)
© Fraunhofer
Medical Knowledge Through Research
What does the neural network see?
![Page 23: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/23.jpg)
© Fraunhofer
Medical Knowledge Through Research
Training
1. Training images with reference labels REF
2. Initialize neural network NN parameters randomly
3. DO
4. Apply NN to a batch of training images → OUTPUT
5. Compute the difference between OUTPUT and REF → LOSS
6. Compute LOSS derivatives w.r.t. NN parameters → GRADIENTS
7. Apply GRADIENTS to update NN parameters
8. UNTIL convergence
![Page 24: DEEP LEARNING ALGORITHMS FOR LIVER AND TUMOR ......Results: MRI Liver Tumor Segmentation n 20 test cases n Automatic method: 0.65 Dice [1] n Human performance: 0.90-0.93 Dice [2] [1]](https://reader034.vdocuments.us/reader034/viewer/2022043008/5f96fc0e645c646fcd53192e/html5/thumbnails/24.jpg)
© Fraunhofer
Medical Knowledge Through Research
Neural network architecture
n U-net like [1]
n 4 resolution levels
n 9M trainable parameters
n Receptive field 94x94 voxels
n 3x3 convolution kernels
n Short skip connections [2]
n Batch normalization
n Spatial dropout[1] Ronneberger O et al., “Convolutional networks for biomedical image segmentation”, MICCAI 2015.
[2] Drozdzal M et al., “The importance of skip connections in biomedical image segmentation”, 2016.