learning to detect photoshopped faces by scripting photoshop · detecting photoshopped faces by...

56
Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang 1 Oliver Wang 2 Andrew Owens 1 Richard Zhang 2 Alexei A. Efros 1 UC Berkeley 1 Adobe Research 2

Upload: others

Post on 18-Oct-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Detecting Photoshopped Faces by

Scripting Photoshop

Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1

Richard Zhang2 Alexei A. Efros1

UC Berkeley1 Adobe Research2

Page 2: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Video Source: https://www.youtube.com/watch?v=5Qqv_C6iVvQ

Page 3: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Warp

detector

Page 4: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Automatic random warping manipulations

Dataset

Page 5: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original

Page 6: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

#1 modification

Page 7: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

#2 modification

Page 8: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

#3 modification

Page 9: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

#4 modification

Page 10: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Real-or-fake classification

99.4%

53.5%

% a

ccura

cy

Ours Human

Page 11: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Dilated ResNet Manipulated Image Warp Prediction

What moved where?

Page 12: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Modified Original Flow Modified Original Flow

What moved where?

Page 13: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Results

Page 14: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated Photo

Page 15: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Flow Prediction

Page 16: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Suggested “Undo”

Page 17: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Photo

Page 18: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated vs. Original

Page 19: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Prediction Manipulated Undo vs. Original

Page 20: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated Photo

Page 21: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Flow Prediction

Page 22: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Suggested “Undo”

Page 23: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Photo

Page 24: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated vs. Original

Page 25: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Undo vs. Original

Page 26: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated Photo

Page 27: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Flow Prediction

Page 28: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “Undo”

Page 29: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Photo

Page 30: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated vs. Original

Page 31: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Prediction Manipulated Undo vs. Original

Page 32: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated Photo

Page 33: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Flow Prediction

Page 34: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Suggested “Undo”

Page 35: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Photo

Page 36: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated vs. Original

Page 37: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Undo vs. Original

Page 38: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated Photo

Page 39: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Flow Prediction

Page 40: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Suggested “Undo”

Page 41: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Photo

Page 42: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated vs. Original

Page 43: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Undo vs. Original

Page 44: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated Photo

Page 45: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Flow Prediction

Page 46: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Suggested “Undo”

Page 47: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Photo

Page 48: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated vs. Original

Page 49: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Undo vs. Original

Page 50: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated Photo

Page 51: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “undo” Prediction Manipulated Flow Prediction

Page 52: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Suggested “Undo”

Page 53: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Original Photo

Page 54: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Manipulated vs. Original

Page 55: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Undo vs. Original

Page 56: Learning to Detect Photoshopped Faces by Scripting Photoshop · Detecting Photoshopped Faces by Scripting Photoshop Sheng-Yu Wang1 Oliver Wang2 Andrew Owens1 Richard Zhang2 Alexei

Thank you!

https://peterwang512.github.io/FALdetector

This work was supported, in part, by DARPA MediFor and UC

Berkeley Center for Long-Term Cybersecurity.

The views, opinions and/or findings expressed are those of theauthors and should not be interpreted as representing the official views or policies of

the Department of Defense or the U.S. Government.