latest developments in image processing on jet by andrea murari 1, j.vega 2, t.craciunescu 3,...
TRANSCRIPT
Latest Developments in Image Processing on JET
by Andrea Murari1, J.Vega2, T.Craciunescu3, P.Arena4, D.Mazon5, L.Gabellieri6, M.Gelfusa7, D.Pacella6, S.Palazzo4, A.Romano6, J.F.Delmond8, A. De Maack9 , T.Lesage8
12
4
5
3
9
87 University of Rome
“Tor Vergata”
6
CODAS: Raw Data
Total Raw data: a record Total Raw data: a record of almost 35 Gbytes per of almost 35 Gbytes per shot has been reached shot has been reached
which keeps JET increase which keeps JET increase in stored information in in stored information in line with the Moore law. line with the Moore law.
JET Database exceeds JET Database exceeds 100 Terabytes100 Terabytes
About 50% are About 50% are images images
Cameras: Visualization
In total more than 30 In total more than 30 cameras operational cameras operational
((PIW protectionPIW protection). ). New visualization tools New visualization tools are indispensable for are indispensable for the analysis (PinUp) the analysis (PinUp)
A new specialist is A new specialist is rostered in the control rostered in the control
room: the VSO (Viewing room: the VSO (Viewing Systems Officer)Systems Officer)
Goals of Imaging in JET
Goals of imaging:
o Imaging of the IR emission from the wall for portection and physics studies
o Imaging of edge instabilities (ELMs, MARFEs etc) for phyics and to assess their effects on the wall.
o Overview of the general discharge behaviour
Issues of Imaging in JET
Issues posed by the exploitation of images:
o Information retrieval (discussed in detail last meeting)
o Image registration (vibrations and interference)
o Integration of models (see V.Martin Talk)
o Real time identification of events
o Extraction of quantitative information for physics studies (see T.Craciunescu Talk)
Mathematical indicators
• 8 different mathematical indicators for vibration detection have been investigated:
• Normalized cross-correlation• Shannon entropy• Tsallis entropy• Renyi entropy• Alpha entropy• Shannon mutual information• Tsallis mutual information• Renyi mutual information
Entropy
Mutual information
Normalized cross-correlation
Normalized cross-correlation
Shannon Entropy Shannon Entropy
Tsallis entropy / Sq entropy Tsallis entropy / Sq entropy
q : degree of non-additivity q : degree of non-additivity
Equal when q 1
pi : probability of finding the system in each possible state i (or residual i)k : Total number of possible states(or number of possible residuals)
Additive and Non additive entropy
Shannon entropy is additive because it assumes that there are no correlations between the systems being added
Tsallis entropy is not additive because it can take into account these correlations.
Tsallis entropy is not additive. For a sum of two systems A1 and A2
Tsallis entropy is finding many applications from statistical mechanics to signal processing, image processing etc
Sq (A1 + A2 ) = Sq (A1) + Sq (A2) + (1-q) Sq (A1) Sq (A2)
In the case of camera movements, the difference between two frames presents long range correlations
These long range correlations, which are less pronounced, in case of objects moving in the still field of view of a camera, can be emphasised by the proper selection of q in the Tsallis entropy.
Application of Tsallis entropy to image registration
Shannon Entropy : 0 Sq entropy : 0 Shannon Entropy : 0 Sq entropy : 0 Shannon Entropy :
0.61 Sq entropy :
3.16
Shannon Entropy : 0.61
Sq entropy : 3.16
Shannon Entropy : 0.81
Sq entropy : 3.99
Shannon Entropy : 0.81
Sq entropy : 3.99
Background
Matrix
Background
Matrix
Object Matrix q=0.1 Object Matrix q=0.1
Shannon Entropy : +0.23
Sq entropy : +0.83
Shannon Entropy : +0.23
Sq entropy : +0.83
Red: Tsallis entropy versus row shift
Blue: Shannon entropy vs row shift
Mutual information
Renyi definition
The Wide Angle Camera KL7 provides a view of the main vessel in the IR
•The Camera seats at the end of and endoscope with many optical components whose position is not monitored• No reliable reference points in the field of view
Image registration: diagnostic
All the major typical events are included
Plasma current between 2 and 3.5 MA Toroidal field between 1.9 and 3.4T
Statistics of frames observed in JET
• A database of 69 videos and almost 40000 frames has been analysed manually to determine the cases with movements.
Comparison Entropies The vertical lines indicate the period with vibrations
Comparison Mutual Informations
Statistics: Threshold• Method: determination of a threshold discriminating
between the frames with and without movements
No mouvement
No mouvement
MouvementMouvement
Succes Rate: OverviewConclusions
Threshold % of good results
Frame where no movement is wrongly detected
Frame where movement is wrongly detected
Normalized cross-correlation 0.94 71.66 14.84 3.78
Shannon entropy 1.6 84.17 15.35 0.48
Shannonmutual information 0.62 78.09 0.47 21.44
Tsallis entropy 25 86.19 6.66 7.15
Tsallismutual information 0.58 79.98 0.48 19.54
Renyi entropy 8 84.70 15.14 0.16
Renyimutual information 1.28 79.80 2.58 17.62
• The result is that entropy of Tsallis is the best among the other entropies.• The mutual information with Tsallis definition is the best definitions among from the definition of mutual information and NCC.
Success Rate: missed and false alarms
86,19%
6,66%7,15%
Tsallis entropy analysis
Correct analysis
Frame where no mouvement is wrongly detected
Frame where mouvement is wrongly detected
False alarmsMissed alarms
Succesfull identifications
Registration: Method Comparison
• A synthetic videos has been shifted by 10 rows and then two of the best indicators have been tried to register it.
Shift
Application to video 73851, frame 786
• Frame 786 is chosen among frames with vibrations. The result of the Tsallis mutual information, which is shown below, is the matrix must be shift by two rows leftwards.
Verification
Mean(value of pixel)=1.3864
Mean(value of pixel)=1.2788
• Subtraction of the frame affected by the movement and the reference frame before and after the registration shows a clear improvement. More effective in the main chamber because the divertor is affected by ELMs
Image Analysis: Hot spot detection
The white areas represent the potential hot regions, parts of the wall which reach a to high temperature.
11,300 frames have been analysed manually
A C++ algorithm to be run on a serial machine has been developed to automatically identify the hot spots (100% success rate in terms of image processing not physics)• Infrared Wide Angle
View: Size of IR images: 496x560 pixels Assumption: the
temperature map provided is correct
Reference serial algorithm: computational time
• For traditional serial algorithms, the computational time depends on the content of the image. A potential problem for real time applications
Computational time versus number of white pixels
Computational time evolution during a discharge
• Array of cells– Information for each cell:
• State (mapped to greyscale value)• Input• Output (dependent on state)
– Each cell is connected to a set of neighbours (usually belonging to a 3x3 square)
– A state equation defines the time evolution of the cell:
Cellular Nonlinear Networks
ijjiSlkC
kljiSlkC
klijij zulkjiBylkjiAxxrr
),(),(),(),(
,;,,;,
where xij is the state of the cell, ykl the output and ukl the input.
• CNNs are a new computational paradigm. If supported by an adequate memory they have the same computational power of Universal Turing machines but with the benefit of parallelism.
• A, B: feedback and input synaptic operators– They define how the state evolves and how neighbour
cells influence it.
– For image processing, they define the kind of filter implemented by the CNN, and are usually 3x3 matrices
a-1,-1 a-1,0 a-1,1
a0,-1 a0,0 a0, 1
a1,-1 a1,0 a1,1
ijjiSlkC
kljiSlkC
klijij zulkjiBylkjiAxxrr
),(),(),(),(
,;,,;,
• zij is a bias constant.
• The set (A, B, z) is called a template. Nonlinear (morphological) operators can be implemented
yi-1,j-1 yi-1,j yi-1,j+1
yi,j-1 yi,j yi,j+1
yi+1,j-1 yi+1,j yi+1,j+1
Summation of dot products
Cellular Nonlinear Networks
1. Directed Growing Shadow
• This template create “shadows” from white pixels by increasing the objects. The template was customized so that the main direction of growth is horizontal.
This template allows merging small close regions – this corresponds to the clustering operation of the serial algorithm.
To be classified as hot spot
To be eliminated
2. ConcaveFiller
• The ConcaveFiller template is applied in order to avoid that the following shrinking phase might separate the regions unified by DirectedGrowingShadow.
S.Palazzo, A.Murari et al REVIEW OF SCIENTIFIC INSTRUMENTS 81, 083505 2010
3. Object Decreasing
• Object Decreasing is applied in order to rescale the objects back to their original size, while keeping the merge regions united.
• Object Removal allows to remove “small objects”
How to implement different processing algorithms to different parts of the images?
Space-varying CNNs
• The implementation approach is based on the definitions of regions in the input image.
• The image is divided into a grid of rectangular cells (regions), by specifying the coordinates of the grid’s rows and columns.
• Each region is then assigned its own sequence of templates, which can differ from other regions in terms of number of templates to be applied, number of iterations or templates’ coefficients. Mathematics already developed.
• The total computation time will depend on the longest template sequence among all regions.
CNN implementation on FPGA
Core array architecture A core takes as input a stripe of the
image (or the output of the upper-row core) and computes the next iteration.
All cores in a column process the same part of the image.
All cores in a row execute the same iteration (on different input stripes).
Parallelism is provided by adding columns to the array – that is, by dividing the image into more parts, to be independently processed.
Hot spot detection• The new algorithm divides the image
into different number of regions on which it is possible to:– Apply customized temperature thresholds,
for example a higher one in the bottom-left divertor’s region.
– Apply region-specific template sequences, in order to improve the global detection accuracy.
Deterministic computational time Implementation with FPGA using cores Total computation time with a 100 MHz clock and 1 column of cores:
106 ∙10 ns = 10 ms → Maximum frame rate: 100 fps
It is possible to increase the frame rate by adding parallelism, i.e. more columns in the core array architecture. With a 10-column core array, the computation time is reduced to 1 ms, and the maximum input frame rate becomes 1000 fps.
Conclusions
o Bidimensional measurements are the new frontier in plasma physics (they are a step forward comparable to profiles)
oVideos contain a wealth of information which can give a very significant contribution to both the understanding of the physics and the real time control of fusion plasmas (including protection)
o Image manipulation: many tools are on the market but they are not always exactly what is needed and therefore significant level of development is required
Apha entropy
Tsallis definition
Results• The figure below shows the Iα entropy. This
entropy does not provide coherent and understable results so it will not be used in the following.