gpu-enabled ultrasound imaging - nvidiaon-demand.gputechconf.com/gtc/2018/presentation/s8764... ·...

Dr. Christoph Hennersperger

Research Manager, Technical University of MunichResearch Fellow, Trinity College Dublin

CTO, OneProjects

GPU-Enabled Ultrasound ImagingReal-Time, Fully-Flexible Data Processing

1/27/2016 | Slide 2GPU-Enabled Ultrasound Imaging | Christoph Hennersperger

Manual Navigation

Reliability on Operator

Complex Diagnostics


Flexible Acquisition

Guidance by/for Expert

Intelligent Imaging

Dr. Christoph Hennersperger

Research Manager, Technical University of MunichResearch Fellow, Trinity College Dublin

CTO, OneProjects

Real-Time, Fully-Flexible Data Acquisition

Data Processing Toward Improved Diagnostics

1/27/2016 | Slide 5

Brief Background on Ultrasound Imaging Workflow

Delay & Focus

Carrier signal

Shape signal

Pulse shape

Active elements

Beamforming Electronic delays for focusing Applicable in transmit and

receive

GPU-Enabled Ultrasound Imaging | Christoph Hennersperger

B-mode image

ScanlineScanline RF data

Scanline envelope

Scanline

Processing of received data Radiofrequency data (RF)

scanline signals Envelope detection

(Hilbert transform) Subsampling (decimation)

Postprocessing for visualization Subsequent filters pipeline

(e.g. speckle reduction) Reduction of dynamic range

(Log-compression) Transformation of scanlines to

image (Scan-conversion)

1/27/2016 | Slide 6

The Need for a Software Defined Ultrasound Framework

Most ultrasound systems have limited flexibility Implementation of major processing on DSPs, FPGAs or ASICs Change of specific points require significant changes

Most ultrasound systems are closed systems Access to images only through PACS Proprietary access and interfaces


Not usable for fast prototyping or research

1/27/2016 | Slide 7

Software Defined Ultrasound Platform for Real-time Applications

Mission: Provide framework to allow covering research aspects from low-level US to high-level applications

Key Design Properties Data and module-driven approach Fully software-defined platform (with GPU)


Fully flexible design Fast prototyping Transparent storage Fully real-time

1/27/2016 | Slide 8

General Ultrasound Processing Layout


Transmit Beamforming(Aperture, Delays) Transmit & Receive

Receive Beamforming (Delay and Sum

with Apodization)

Envelope detection incl. Frequency Compounding Log-Compression Scan-Conversion

GPU accelerated

Controllable via

1/27/2016 | Slide 9

Beamforming on GPU

Parallelization for individual scanlines and samples Aperture defines input data (number of channels)

Delay and sum beamformer in SUPRA

Blocks operate on receive scanlines Blocks process individual rx scanlines Shared memory over local aperture (memory access)

Thread operate on receive samples Individual threads process samples of scanlines Local thread performs DAS over aperture (x,y) and depth (z) using

shared memory


+

τ2τ1 τ3 τ4 τ5 τ6 τ7 τ8

1/27/2016 | Slide 10

Imaging with SUPRA


Fast Acquisitions

Storage of Full Data

Direct Configuration

1/27/2016 | Slide 11

Qualitative Evaluation to Proprietary Scanline Imaging


Cep

haso

nics

SUP

RA

Point Phantom Muscle Fibers Carotid Transverse Carotid Longitudinal

1/27/2016 | Slide 12

From Scanline to Planewave Imaging

64 scanlines Max 300 Hz

64 Angles 300 Hz

10 Angles 1925 Hz


Scanline Planewave Planewave

1/27/2016 | Slide 13

Real-time Capabilities of Framework

Results for NVIDIA Jetson TX2, mobile GTX and GTX 1080



Towards Improving Diagnostic Outcomes with US

1/27/2016 | Slide 15

Ultrasound - Unique Abilities and Challenges


Data interpretation Image Graph (network) Continuous signals

1/27/2016 | Slide 16




Understanding of physics Signals from reflection Acoustic tissue properties

1/27/2016 | Slide 17




Understanding of physics Signals from reflection Acoustic tissue properties

Challenges and artefacts Shadowing and enhancement Nonlinearity of tissue propagation Interference of waves

1/27/2016 | Slide 19

Modeling Ultrasound as Arbitrarily Sampled Data


• Overlapping US-slices• Resampling to regular grid


• Overlapping US-slices• Resampling to regular grid• Process on regular graph

Loss of information regarding acquisition!


1/27/2016 | Slide 21



Better: Define graph on original samples

• Graph nodes represent US samples• Graph edges represent spatial

structure

Construct edges in “local” coordinates

1/27/2016 | Slide 23





structure


1) Zu Berge, C. S., Declara, D., Hennersperger, C., Baust, M., & Navab, N. (2015, October). Real-time uncertainty visualization for B-mode ultrasound. IEEE SciVis 2015.2) Hennersperger, C., Mateus, D., Baust, M., & Navab, N. (2014, September). A quadratic energy minimization framework for signal loss estimation from arbitrarily sampled ultrasound data, MICCAI 20153) Virga, S., Zettinig, O., Esposito, M., Pfister, K., Frisch, … & Hennersperger, C. (2016, October). Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms, IEEE IROS 2016

1/27/2016 | Slide 24





structure


1) Virga, S., Göbl, R., Baust, M., Navab, N., & Hennersperger, C. (2018). Use the force: deformation correction in robotic 3D ultrasound. International journal of computer assisted radiology and surgery, 1-9.

1/27/2016 | Slide 25

Connecting the Dots

Improving Diagnostic Outcomes Domain specific knowledge to improve

diagnostic benefit Data science approaches with need for

data (end to end)


SUPRA as enabling technology• Fully open, flexible, and real-time• Tool for rapid demonstration and R&D

1/27/2016 | Slide 26

Next: Try it Yourself!


Even without an ultrasound machine: https://github.com/IFL-CAMP/supra

Rüdiger Göbl

Christoph Hennersperger

Nassir Navab

This project received funding from the European Union’s H2020 research and innovation programme (No 688279

gpu-enabled ultrasound imaging - nvidiaon-demand.gputechconf.com/gtc/2018/presentation/s8764... ·...

Documents