1 how realistic is photorealistic?. 2 yaniv lefel hagay pollak based on the work of - siwei lyu and...
Post on 20-Jan-2016
216 views
TRANSCRIPT
![Page 1: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/1.jpg)
1How
Rea
listic
is P
hoto
real
istic
?
![Page 2: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/2.jpg)
2
How Realistic is Photorealistic?
Yaniv Lefel
Hagay Pollak
Based on the work of - Siwei Lyu and Hany Farid
![Page 3: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/3.jpg)
3
Introduction
• Among the set of all possible images, natural images only occupy a tiny subspace.
• For instance, there are totally 256^(n^2) different 8-bit grayscale images of size nxn pixels. Natural images are sparsely distributed in the space of all possible images.
![Page 4: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/4.jpg)
4
Image space
•e.g. when n = 10 pixels, it results in 1.3x10^154 different images !!!
![Page 5: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/5.jpg)
5
Introduction (cont’)
• The regularities within natural images can be modeled statistically.
• Image statistical models are already in use by applications such as:Compression, de-noising, segmentation, texture synthesis, content-based retrieval and object/scene categorization.
![Page 6: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/6.jpg)
6
Motivation 1 Identify Computer Graphics
• Sophisticated computer graphics software can generate highly convincing photorealistic images able to deceive the human eye.
• Differentiating these two types of images is an important task to ensure the authenticity and integrity of photographs.
![Page 7: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/7.jpg)
7
Computer graphics example
![Page 8: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/8.jpg)
8
Motivation 2Identify Steg Images
• Image steganography hides messages in digital images in a non-intrusive way that is hard to detect visually.
• The task of generic steganalysis is to detect the presence of such hidden messages without the detailed knowledge of the embedding methods.
![Page 9: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/9.jpg)
9
Steganography example
• Steg is the message image embedded into the original image.• The rightmost image is the absolute value of the difference
between the original and steg image, normalized into 8 bit for display purposes.
Original
message
Steg |Original-Stego|
![Page 10: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/10.jpg)
10
How ? Example
![Page 11: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/11.jpg)
11
Motivation 3Identify Re-broadcasting
• Biometrics-based (e.g., face, iris, or voice) authentication and identification systems are vulnerable to the “rebroadcast” attacks. (e.g. using a high-resolution photograph of a human face).
• We need to differentiate a “live” image (captured in real time by a camera) and a “rebroadcast” one (a photograph).
![Page 12: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/12.jpg)
12
How to distinuish images ?
• Image properties ? – Image intensity histogram– Image frequency
Other method ?
![Page 13: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/13.jpg)
13
Using known methods
![Page 14: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/14.jpg)
14
Why wavelets
• Image representations based on multi-scale image decomposition (e.g., wavelets) decompose an image with basis functions partially localized in both space and frequency - a compromise between these representations.
![Page 15: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/15.jpg)
17
QMF - Quadrature Mirror Filter
• The QMF pyramid decomposition splits the image frequency space into three different scales, and within each scale, into three orientation subbands (Vertical, Horizontal and Diagonal).
![Page 16: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/16.jpg)
18
QMF diagram
![Page 17: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/17.jpg)
19
QMF
• The vertical, horizontal and diagonal subbands at scale i are denoted by Vi(x; y), Hi(x; y), and Di(x; y), respectively.
• Can be generated by convolving the image, I(x, y), with low-pass and high-pass filters.
![Page 18: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/18.jpg)
23
QMF decomposition – Example 1
![Page 19: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/19.jpg)
24
QMF decomposition – Example 2
![Page 20: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/20.jpg)
25
Example – QMF statistics
![Page 21: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/21.jpg)
28
Add some magic …• QMF
coefficients
• Magic Box
• Error coefficients
Simple but long (and out of scope)
mathematical procedure
![Page 22: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/22.jpg)
32
Technique Diagram
Feature vector
![Page 23: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/23.jpg)
33
Computing the Feature Vector
• 3 – Sub-bands (vertical, horizontal, diagonal).• 3 – Scales (levels of decompositions).• 4 – First order statistics (mean, variance, skewness
– asymmetry measure, kurtosis).• 3 – Colors (RGB)• 2 – marginal statistics (wavelet coefficients),
error statistics.• 216 = 3*3*4*3*2
![Page 24: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/24.jpg)
34
Image examples
![Page 25: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/25.jpg)
35
Feature vectors projected on 3D space
Natural image – Blue.Synthetic images - noise (Green), fractal (Black), and discs (Red)
![Page 26: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/26.jpg)
36
Learning and Testing CG\Steg\rebroadcast
• CG\steg\rebroadcast images are prepared.• Statistics is collected over natural images and
CG\steg\rebroadcast images (not using color).• A Machine learning system (e.g. FLD, LDA,
SVM) is then trained on some of the natural and some of the CG\steg\rebroadcast images.
• The remaining natural and CG\steg\rebroadcast images are used for testing.
![Page 27: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/27.jpg)
37
Natural vs. CG results (SVM)
All images
Train Succ [%]
Test Succ [%]
Natural 40000 32000 70.9 8000 66.8
CG 6000 4800 99.1 1200 98.8
![Page 28: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/28.jpg)
38
Training the system
![Page 29: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/29.jpg)
39
Photorealistic (CG) images
![Page 30: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/30.jpg)
41
The Impact of Color
![Page 31: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/31.jpg)
42
Correctly Classified Photorealistic
![Page 32: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/32.jpg)
43
Incorrectly Classified Photographic
![Page 33: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/33.jpg)
44
Natural vs. Steganography images
• A message consists of a 64x64 pixel region of a random image chosen from the same image database.
![Page 34: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/34.jpg)
45
Natural vs. Steganography results
All images
Train Succ [%]
Test Succ [%]
Natural 1000 750 99.5 250 98.9
Steg 1000 750 98.3 250 97.6
![Page 35: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/35.jpg)
46
Live vs. rebroadcast
• We collect statistics from natural images and the same images after having been printed on a laser printer and re-scanned with a scanner (printing and scanning are done at 72 dpi).
![Page 36: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/36.jpg)
47
Live vs. rebroadcast results
All images
Train Succ [%]
Test Succ [%]
Natural 1000 750 99.5 250 99.5
rebroadcast
200 150 100 50 99.8
![Page 37: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/37.jpg)
48
Live vs. rebroadcast (cont’)
• Remark: It is not surprising that printing significantly disturbs the image statistics. Detecting a rebroadcast image will become more difficult with printers improvement.
![Page 38: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/38.jpg)
49
Rebroadcasting example
Shown is the original iris images (top row) and the images after being printed and scanned (bottom row).
![Page 39: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/39.jpg)
50
Feature vectors projected on 3D space
Results from a four-way classifier of 1000 natural, 1000 steg, 500 graphic, and 200 rebroadcast images.
![Page 40: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/40.jpg)
51
More Applications
how many different artists ?
![Page 41: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/41.jpg)
52
More Applications
Forgery detection.
![Page 42: 1 How Realistic is Photorealistic?. 2 Yaniv Lefel Hagay Pollak Based on the work of - Siwei Lyu and Hany Farid](https://reader035.vdocuments.us/reader035/viewer/2022062806/56649d5e5503460f94a3e54a/html5/thumbnails/42.jpg)
54
Finally
• Statistical model.
• capture regularities that are inherent to photographic images.
• Distinguish tampered \ CG images and natural images.