![Page 1: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/1.jpg)
Model Inversion Attacks AgainstCollaborative Inference
Zecheng He1, Tianwei Zhang2, and Ruby B. Lee1
1Princeton University2Nanyang Technological University
1
![Page 2: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/2.jpg)
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
2
![Page 3: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/3.jpg)
q Deep learning is everywhere• Computer Vision, Natural Language Processing, Robotics,
Medical, Autonomous Driving…
q Cloud-based deep learning systems become prevalent• Google Cloud AI, Amazon Sagemaker, Microsoft Azure AI…• Collaborative Training, Collaborative Inference
Motivation
3
![Page 4: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/4.jpg)
Collaborative Training
Parameter server
Client
…Client
4
g2t
𝜽𝒕
gnt
𝜽𝒕
𝜽𝒕
gn-1t
Client
Client
![Page 5: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/5.jpg)
Collaborative Inferenceq Split the DNN to multiple parties
• Hybrid IoT / Edge-Cloud Computation• Easy parts computed on the edge device, hard parts on the cloud• Save power and reduce latency
Cloud
Edge Device
Edge Device Edge Device
Edge Device
![Page 6: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/6.jpg)
Attacks against Training Data Privacy
q Model inversion attacks• Recover representative input of each class• ‘Average’ of data instead of a specific input. • Not work well for deep NNs
q Membership attacks• Whether a given sample is in the training set?• Need to know candidate training samples
q Attribute/property attacks• Whether the training data has a property, e.g. color, gender• Obtain property information, not an individual sample
6
![Page 7: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/7.jpg)
q Inference data privacy is less studied• More severe problem:
ü Unlike fixed number of training data, the number of inference data is increasing over time
ü Inference samples could be more sensitive
• More challenging problem:ü Trained model does not depend on inference dataü Inference samples vary significantly
No Attacks against Collaborative Inference
Is it possible to recover useful information of individual inference data input?
7
![Page 8: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/8.jpg)
Threat Model
q Adversary: Untrusted or compromisedcloud provider
q Target:Recover sensitive inputs during inference
q Adversary’s capabilities1) White-box of the edge-side model2) Black-box of the edge-side model3) No query to the edge-side model
Edge-side model Cloud-side model
Attack works in all scenarios!8
![Page 9: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/9.jpg)
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
9
![Page 10: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/10.jpg)
?0.850.020.410.19...
[
[
White-box Attack
Intermediate values𝑓$% 𝑥'
Insight: find an input x, such that1) 𝑓$% 𝑥 is close to 𝑓$% 𝑥'2) Domain knowledge: 𝑥 is a natural input
10
Known parameters 𝑓$%
![Page 11: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/11.jpg)
White-box Attack
Regularized Maximum Likelihood Estimation (rMLE)
1) 𝐟𝛉𝟏 𝐱 is close to 𝐟𝛉𝟏 𝐱𝟎 => Euclidean Distance (ED)
2) 𝐱 is a natural sample => Small Total Variation (TV)
ED(𝑥, 𝑥') = ||𝑓$% 𝑥 − 𝑓$% 𝑥' ||88
TV(𝑥) = ;(|𝑥<=%,> − 𝑥<,>|8 + |𝑥<,>=% − 𝑥<,>|8)@/8<,>
𝒙∗ = 𝒂𝒓𝒈𝒎𝒊𝒏𝒙 𝑬𝑫 𝒙,𝒙𝟎 + 𝝀𝑻𝑽(𝒙)
S𝐨𝐥𝐯𝐞 𝒙 with Gradient Descent11
![Page 12: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/12.jpg)
White-box Evaluation
Dataset MNIST CIFAR10Target Model 2 conv + 3 fc (LeNet5) 6 conv + 2 fcSplit Point • 1st conv layer (conv1)
• 2nd conv layer after activation (ReLU2)
• 1st conv layer (conv11)• 4th conv layer after
activation (ReLU22)• 6th conv layer after
activation (ReLU32)
q Target models and datasets
12conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2
![Page 13: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/13.jpg)
White-box Evaluation
13
![Page 14: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/14.jpg)
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
14
![Page 15: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/15.jpg)
?0.850.020.410.19...
[
[
𝑓$% 𝑥'
Black-box Attack
Insight: Train 𝑓$%S%, an ‘inverse network’ of 𝑓$%, 𝑚𝑎𝑝𝑝𝑖𝑛𝑔
𝑓$% 𝑥 back to x
The adversary can query 𝑓$%
15
Known parameters 𝑓$%
![Page 16: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/16.jpg)
Black-box Attack
q Inverse Network 𝒇𝜽𝟏S𝟏1) Generate training samples2) Train the inverse network
𝑓$%S% = 𝑎𝑟𝑔𝑚𝑖𝑛\
1𝑚; ||𝑔 𝑓$% 𝑥< − 𝑥<||8]
<^%
𝑓$% 𝑥% , … , 𝑓$% 𝑥] ~(𝑥%,… , 𝑥])
𝑓$% 𝑓$%S%
16
![Page 17: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/17.jpg)
Black-box Attack Evaluation
17
![Page 18: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/18.jpg)
Black-box Attack Evaluation
q Is knowing training data / distribution important?
1) Train Inverse Network with same training data
2) Train Inverse Network with data from the same distribution
3) Train Inverse Network with random noise
𝑓$%S% = 𝑎𝑟𝑔𝑚𝑖𝑛\
1𝑚; ||𝑔 𝑓$% 𝑥< − 𝑥<||8]
<^%
18
![Page 19: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/19.jpg)
Black-box Attack Evaluation
q Is knowing training data / distribution important?
Inverse Network trained with the same training data
Original inputs
Inverse Network trained with the same training distribution
Inverse Network trained with random noise 19
![Page 20: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/20.jpg)
Black-box Attack Evaluation
q The data distribution is important• Random noise to train the inverse network can only
partially recover the inputs
20
q The exact training samples are not necessary
q Quantitative results in the paper• Peak Signal Noise Ratio (PSNR)• Structural Similarity (SSIM)
![Page 21: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/21.jpg)
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
21
![Page 22: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/22.jpg)
?0.850.020.410.19...
[
[
𝑓$% 𝑥'
Access-free Attack
Known parameters 𝑓$%The adversary can query 𝑓$%
Insight: Reconstruct a shadow model 𝑓′$% , then perform a white-box attack on 𝑓′$%
22
![Page 23: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/23.jpg)
Shadow Model Construction
The cloud-side model 𝑓$8, frozen
The shadow (edge-side) model 𝑓′$% , trainable
Cat
𝑓′$% = 𝑎𝑟𝑔𝑚𝑖𝑛\ ;CrossEntropy(𝑓$8 𝑔 𝑥< , 𝑦<) ]
<^%
𝑓′$% and 𝑓$8 (frozen) jointly perform well on the original classification task
23
![Page 24: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/24.jpg)
White-box Attack on the Shadow Model
ED′(𝑥, 𝑥') = ||𝑓′$% 𝑥 − 𝑓$% 𝑥' ||88
𝒙∗ = 𝒂𝒓𝒈𝒎𝒊𝒏𝒙 𝑬𝑫′ 𝒙, 𝒙𝟎 + 𝝀𝑻𝑽(𝒙)
0.850.020.410.19...
[
[
𝑓′$% 𝑥'𝑓′$%
White-box rMLE attack against 𝑓′$%
24
![Page 25: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/25.jpg)
Access-free Attack Evaluation
25
![Page 26: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/26.jpg)
Access-free Attack Evaluationq Is knowing exact training data important?
26
Original input
Same training data
Same distribution
Original input
Same training data
Same distribution
![Page 27: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/27.jpg)
Where should the model be split between edge and cloud?
PSRN and SSIM of query-free attackagainst v.s. split points
Query-free attack against v.s. split points
27
![Page 28: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/28.jpg)
Model Inversion Attacks AgainstCollaborative Inference
•Motivation and Background•White-box Attack• Black-box Attack• Access-free Attack• Defense and Conclusion
28
![Page 29: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/29.jpg)
Defenses
q Running FC layers before sending to the cloud, or make the edge-side network deeper• High overhead on edge-side
q Homomorphic Encryption• Too complex, not practical
q Differentially private model inference• Adding noise to the inference input: 𝑣 = 𝑓$%(𝑥 + 𝜀)• Adding noise to the intermediate value: 𝑣 = 𝑓$% 𝑥 + 𝜀• Tradeoff between accuracy (usability) and privacy
29
![Page 30: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/30.jpg)
Conclusion
q Three new attacks in collaborative inference scenarios• White-box attack• Black-box attack• Query-free attack• Adversary can successfully recovers sensitive inputs• Exact training data is not important to the success of the attack, but
training data distribution is important
q Data privacy should be considered in collaborative inference system design
30
![Page 31: Model Inversion Attacks against Collaborative Inference v2 · conv11 conv12 pool1 conv21 conv22 pool2 conv31 conv32 pool3 FC1 FC2. White-box Evaluation 13. Model Inversion Attacks](https://reader036.vdocuments.us/reader036/viewer/2022081611/5f026a9a7e708231d4042b14/html5/thumbnails/31.jpg)
Code available
https://github.com/zechenghe/Inverse_Collaborative_Inference
Questions?
31