gray image coloring using texture similarity measures
TRANSCRIPT
1
Gray Image Coloring Using Texture Similarity Measures
by
E. Noura Abd El-Moez E. Noura Abd El-Moez SemarySemary
Thesis Submitted in accordance with the requirements of Thesis Submitted in accordance with the requirements of The University of Monofiya for the degree ofThe University of Monofiya for the degree of
Master of Computers and InformationMaster of Computers and Information( Information Technology )( Information Technology )
Gray Image Coloring Using Texture Similarity Measures
Presented by
E. Noura Abd El-Moez E. Noura Abd El-Moez SemarySemary
For Master degree in Computers and InformationIT department, Faculty of Computers and information,
Menofia University
Supervised by:Prof.
Mohiy .M.Hadhoud
Prof. Nabil .A.Ism
ail
Dr.Waiel .S.Al-Kilani
Thesis summary on:
الرمادية الصور تلوينتشابه معايير بإستخدام
األنسجة
من :مقدم
المعز. عبد نورا المعز. م عبد نورا مسمري سمري السباعي السباعي
في الماجستير درجة على والمعلومات للحصول الحاسبات المعلومات تكنولوجيا المعلومات - - قسم و الحاسبات كلية
المنوفية جامعة
شوقي. وائل دالكيالني
. عبد. نبيل د أالواحد
إسماعيل
. محمد. محي د أهدهود
إشراف :تحت
بعنوان رسالة :ملخص
4
Outlines
Introduction Automatic coloring in the literature TRICS ‘Texture Recognition based Image
Coloring System’ Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
5
Introduction Gray image principles
Gray values
0 . .5
0 . .1
00 . .
15
0 . .2
00 . .
25
0 .2
55
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
6
Introduction Gray image principles
21.2 kb246256170 ×120 px
60.0 kb15,99816,777,216170 ×120 px
Total size (byte)
Actually No. used colors
Possible No. colors
Image Size (pixel)
Image
7
Introduction Coloring Problem
There are two definitions to describe the gray value as an equation of the three basic components of RGB color model (red, green, blue):
1: Intensity (most common used)Gray = (Red + Green + Blue) /3
2: Luminance (NTSC standard for luminance)Gray = (0.299 × Red) + (0.587 × Green) + (0.114 × Blue)
RGB Color R, G, B values Gray value Gray Color
100 ,150 ,87 128
147, 87, 149 128
149, 147, 87 128THERE IS NO MATHEMATICAL FORMULA TO CONVERT FROM GRAY TO RGB COLOR.
8
Introduction Coloring Problem
HSL Color wheel Grayed Color Wheel
Similar Gray ValuesSimilar Gray Values
9
Introduction Coloring Types
1 . Hand coloring Adobe Photoshop
and Paintshop Pro Layers Changing Hue
BlackMagic, photo colorization software, version 2.8, 2003
10
Introduction Coloring Types
2 . Semi automatic coloring Pseudocoloring is a
common example for semi automatic coloring technique
11
Introduction Coloring Types
3 . Automatic coloring i. Transformational coloring
ii. Matched image coloring
iii. User selected coloring
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
12
Automatic coloring in the literature
1. Transformational Coloring
A transformation function Tk is applied on the intensity value of each pixel Ig(i,j) resulting in the chromatic value Ick(i,j) for channel k
)],([),( jiITjiI gkkc
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
13
Automatic coloring in the literature
1. Transformational Coloring
Al-Gindy et al * system. × Results have unnatural look
* A. N. Al-Gindy, H. Al Ahmad, R. A. Abd Alhameed, M. S. Abou Naaj and P. S. Excell ’Frequency Domain Technique For Colouring Gray Level Images’ 2004 found in www.abhath.org/html/modules/pnAbhath/download.php?fid=32
14
Automatic coloring in the literature
2. Matched image coloring
The most similar pixel color is transferred to the corresponding gray one by the color transfer technique proposed by E.Reinhard*;
* Reinhard, E. Ashikhmin, M., Gooch B. And Shirley, P., Color Transfer between Images, IEEE Computer Graphics and Applications, September/October 2001, 34-40
ll αα ββ
15
Automatic coloring in the literature
2. Matched image coloring
* T. Welsh, M. Ashikhmin, K. Mueller. “Transferring color to greyscale images.” In Proceedings of the 29th Annual Conference on Computer Graphics and interactive Techniques, pp 277–280, 2002
** Y. Tai, J. Jia, C. Tang ‘Local Color Transfer via Probabilistic Segmentation by Expectation- maximization‘, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Volume 1, pp. 747-754, 2005
“Global matching procedure” of T. Welsh et al* “Local color transfer” of Y. Tai et al**.
× All these algorithms fail, when different colored regions have similar intensities
16
Automatic coloring in the literature
2. Matched image coloring
Welsh et al proposed also another technique to improve the coloring results when the matching results are not satisfying. It was achieved by asking users to identify
and associate small rectangles, called “swatches” in both the source and destination images to indicate how
certain key colors should be transferred
17
Automatic coloring in the literature
3. User selection coloring
User selection coloring gives high quality colors× User dependent color quality× Time-consuming× Colorization must be fully recomputed for any slight change in the
initial marked pixels
* A. Levin, D. Lischinski, Y. Weiss. “Colorization using optimization.” ACM Transactions on Graphics, Volume 23, Issue 3, pp.689–694, 2004
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
18
TRICS System
Research Objectives
To simulate the human vision in coloring process
To be fully automatic coloring system To spend so little execution time as possible
as a basic requirement for video coloring.
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
19
TRICS System
Structure Gray image
Segmentation
Classification
Database
Samples features
Classes HuesClass labels
Coloring
Colored image
Segmented image, Clusters
Features extraction(Joint, wavelets, laws,…)
Segmentation
(Mean Shift, K-Mean, FCM,..)
1 2
Features extraction(Co-occurrence, Tamura, Wavelets energies)
Classification(K-NN classifier,..)
43
Convert image to HSV channels
Convert to RGB
Set Hue, Saturation, and Brightness
5 6 7
A
B
C
20
TRICS System
Structure 1. Segmentation Stage
Feature extraction: (Pixel based )1. pixel position
2. pixel intensity
3. texture features wavelets coefficients Laws kernels coefficients.
21
TRICS System
Structure 1. Segmentation Stage
1. Wavelets coefficients× Quarter the image size.
Up sampling Upper level construction
22
TRICS System
Structure 1. Segmentation Stage
Up sampling
Upper level construction
23
TRICS System
Structure 1. Segmentation Stage
2. Laws Kernels :
Level L5 = [ 1 4 6 4 1]Edge E5 = [ -1 –2 0 2 1]Spot S5 = [ -1 0 2 0 –1]Wave W5= [ -1 2 0 –2 1]Ripple R5 = [ 1 –4 6 –4 1]
L5S5’ = -1 -4 -6 -4 -1
0 0 0 0 0
2 8 12 8 2
0 0 0 0 0
-1 -4 -6 -4 -1
24
TRICS System
Structure 1. Segmentation Stage
Segmentation technique:Mean Shift *K-mean (Fast k-mean) **
Adaptive Fast k-mean
* D. Comaniciu and P. Meer. ‘Mean shift: A robust approach toward feature space analysis.’ PAMI, 24(5):603–619, May 2002
** C.Elkan, ‘Using the triangle inequality to accelerate k-Means.’ In Proc. of ICML 2003. pp 147--153
32
TRICS System
Structure 1. Segmentation Stage
1. Mean Shift :× So slow× Many parameters
(170×256)- hs=16,hr=16,m=500- Time: 0 34 15- classes : 9
- hs=8,hr=8,hw=4,m=500- Time: 0 39 54- classes : 7
34
TRICS System
Structure 1. Segmentation Stage
2. Fast K-mean : with spatial features
× structured segmentation Increase no. clusters
k=3 k=9
35
TRICS System
Structure 1. Segmentation Stage
2. Fast K-mean : without spatial features
× scattered regions of same cluster disjoint region separation
before after
1 1 2
1
3
1 2 3
4
5
36
TRICS System
Structure 1. Segmentation Stage
2. Fast K-mean :× Small regions (noise) Small regions elimination
Original Before After
37
TRICS System
Structure 1. Segmentation Stage
3. Adaptive Fast K-mean : Clusters number generation Minimum region size estimation
Fully automatic segmentation techniqueFully automatic segmentation technique
38
• CEC “Combined Estimation Criterion”*:•If the VRC index, for k clusters, is smaller than 98 of the VRC index, for k-1 clusters, the CEC is not satisfied.
•If the VRC index, for k clusters, is larger than 102 of the VRC index for k-1 clusters, or if k=1, the CEC is satisfied.
•If the VRC index, for clusters, is smaller than 102 but larger than 98 of the VRC index for clusters, the CEC is satisfied only if TSS for k-1 clusters is smaller than 70 of TSS for k clusters.
TRICS System
Structure 1. Segmentation Stage
a. Clusters number generation :
WCSSTSSBCSS
MxTSS
mxWCSS
f
i
n
jiij
f
i
n
jijij
1 1
2
1 1
2
)(
)(
WCSSk
BCSSknVRC
)1(
)(
* D.Charalampidis, T.Kasparis, ‘Wavelet-Based Rotational Invariant Roughness Features for Texture Classification and Segmentation’. IEEE Transactions on Image Processing.Vol.11.No.8 August 2002
39
TRICS System
Structure 1. Segmentation Stage
Fast k-mean
Calculate CEC
CEC Satisfied?
k=k+1
Yes
No
Start k=1
Stop…Clusters number =k
Gray Image
Segmented Image
40
TRICS System
Structure 1. Segmentation Stage
b. Minimum region size estimation :
• Split the disjoint regions.
• Count all regions size.
• Sort regions size and calculate the step between them.
• Select the regions size of step more than the largest image dimension.
• Consider the minimum region size.
41
TRICS System
Structure 1. Segmentation Stage
59 sec
59 sec
EM-time
31 sec
31 sec
E-time
Laws
Waveletssize = 170×256 M=500, EM = 324
10 min
36 sec
3 sec
42 min
12 sec
T- time
F-time
11 sec
19 sec
A-time
7
8
Regions
4
4
ClustersFeaturesImage
Original gray wavelets Laws
A-time : adaptive fast k-mean time, F-time: fixed k fast k-mean time, T-time: traditional k-mean , E-time: elimination time , EM-time : Elimination time with estimating minimum size region
42
TRICS System
Structure Database set1
The training set consists of 32 classes of Brodatz texture database
Each image has a size of 256 × 256. Each image was mirrored horizontally and vertically to produce a 512 ×512 image.
The image is split into 16 images of size 128 ×128.
256 × 256 512 × 512 16 × 128 × 128
43
TRICS System
Structure Database set2
The training set consists of 9 classes ‘cloud, sky, sea, sand, tree, grass, stone, water, and wood’
Each class has number of samples from 12 to 25 samples.
These samples are taken from real natural images as random 64x64 rectangles.
44
TRICS System
Structure Database
Database record: Sample
Class (level1,level2)
58 Features (6 Moment statistics, 4 Co-ocurance measures, 3 Tamura, and “ 15 wavelets mean, 15 wavelets variance , 15 wavelets energy” for five levels wavelets decomposition)
Hue
45
TRICS System
Structure 2. Classification Stage
a. Feature extraction (Region based) Rectangular region:
1. Maximum rectangle
2. 64 x 64 rectangle× Arbitrary shape Padding rectangle*
* Ying Liu, Xiaofang Zhou, Wei-Ying Ma, ‘Extracting Texture Features
from Arbitrary-shaped Regions for Image Retrieval ‘. 2004 IEEE
International Conference on Multimedia and Expo., Taipei, Jun. 2004
46
TRICS System
Structure 2. Classification Stage
a. Feature extraction (Region based) Region based features:
GLCM * measures (Energy, Entropy, Inertia, Homogeneity ) Tamura * (Coarseness, Contrast , Directionality’) Wavelets coefficients for 5 levels
Mean and variance ** Energy ***
* P.Howarth, S.Ruger,: Evaluation of texture features for content-based image retrieval. In: proceedings of the International Conference on Image and Video Retrieval, Springer-Verlag (2004) 326–324
** O. Commowick – C. Lenglet – C. Louchet, ‘Wavelet-Based Texture Classification and Retrieval’ 2003 found in http://www.tsi.enst.fr/tsi/enseignement/ressources/mti/classif-textures/
*** Eka Aulia, ‘Hierarchical Indexing For Region Based Image Retrieval’, Master thesis of Science in Industrial Engineering, Louisiana State University and Agricultural and Mechanical College, May 2005
49
b. Classification technique KNN classifier with (k=1,k=5,k=10,k=20) Distance Metric is L2 “Euclidean distance”
k=5 gives accuracy up to 94% using “N-fold “ (the collection of (sub) images is divided into N disjoint sets, of which N-1 serve as training data in turn and the Nth set is used for testing)
TRICS System
Structure 2. Classification Stage
21
2)()(),(
i
iJiIJIE
50
TRICS System
Structure 2. Classification Stage
51
TRICS System
Structure 3. Coloring Stage
Color model conversion HSV/HSB color model
HSI/HLS color model
Change in SaturationHue = 0, Luminance=0.5
Change in BrightnessHue = 0, Saturation = 1
Change in HueSat=1, Luminance=1
Change in SaturationHue = 0, Luminance=0.5
Change in LuminanceHue = 0, Saturation = 1
Change in HueSat=1, Luminance=1
52
TRICS System
Structure 3. Coloring Stage
HSV & HSL Channels
53
TRICS System
Structure 3. Coloring Stage
Setting Channels valuesBrightness
The gray image itself
Hue One hue value for each texture
Saturation HSV: 1- brightness HSI: 0.5(1-lightness)
54
TRICS System
Structure 3. Coloring Stage
55
TRICS System
Structure 3. Coloring Stage
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
56
Results and Conclusion Results
Perfect Results Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
57
Results and Conclusion Results
Perfect Results
58
Results and Conclusion Results
PANN Database:
59
Results and Conclusion Results
Misclassified results
2 classification levels: •if the KNN results in 5 classes “grass, sea, water, grass, sea”
•The traditional solution is the class of the grass.
•The 2 levels classification solution is sea.
•(Sea and water), (trees and grass), (sky and clouds) and (wood
and stone) are considered as one class in level one.
60
Results and Conclusion Comparisons
HSV/HSB
HIS/HLS
61
Results and Conclusion Comparisons Local color transfer
Global Image Matching
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
62
Results and Conclusion Conclusion We proposed a new computer coloring technique
that simulates the human vision in this area.
The proposed coloring system is contributed for coloring gray natural scenes.
The execution time of TRICS is minimized using Fast k-mean segmentation technique and the results are enhanced by splitting the disjoint regions and by eliminating small regions.
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
63
Results and Conclusion Conclusion
Clusters number generation algorithm and the minimum region size estimation algorithm increase the professionalism of the system but also increases the time of the execution. And by using both of them TRICS becomes a fully unsupervised intelligent recognition based coloring system.
HSV coloring model is very suitable for our system and the coloring results have good natural look.
64
Results and Conclusion Conclusion
We consider our proposed system structure as an abstract structure for building any more intelligent coloring systems for any other types of images
Our proposed system results perform the other coloring systems.
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
65
Future workGray image
Segmentation
Classification
Database
Samples features
Classes HuesClass labels
Coloring
Colored image
Segmented image, Clusters
Features extraction(Joint, wavelets, laws,…)
Segmentation
(Mean Shift, K-Mean, FCM,..)
Features extraction(Co-occurrence, Tamura, Wavelets energies)
Classification(K-NN classifier,..)
Convert image to HSV channels
Convert to RGB
Set Hue, Saturation, and Brightness
A
B
C
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
Outlines
Introduction Automatic
coloring in the literature
TRICS ‘Texture Recognition based Image Coloring System’
Results Conclusion Future work
66
Future workGray image
Segmentation
Classification
Database
Samples features
Classes HuesClass labels
Coloring
Colored image
Segmented image, Clusters
Features extraction(Joint, wavelets, laws,…)
Segmentation
(Mean Shift, K-Mean, FCM,..)
Features extraction(Co-occurrence, Tamura, Wavelets energies)
Classification(K-NN classifier,..)
Convert image to HSV channels
Convert to RGB
Set Hue, Saturation, and Brightness
A
B
C
Intelligent System for Classifying the image
SOFMAdaptive Learning
68
List Of Publications
Noura A.Semary, Mohiy M. Hadhoud, W. S. El-Kilani, and Nabil A. Ismail, “Texture Recognition Based Gray Image Coloring”, The 24th National Radio Science Conference (NRSC2007), pp. C22, March 13-15, 2007, Faculty of Engineering, Ain-Shams Univ., Egypt.
69
Thanks…
و الحمد لله الذي بفضله تتم الصالحات