the optimal mechanism in differential privacy · 11/07/2013 phd final exam of quan geng, ece, uiuc...
TRANSCRIPT
![Page 1: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/1.jpg)
The Optimal Mechanism in Differential Privacy
Quan GengAdvisor: Prof. Pramod Viswanath
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 1
![Page 2: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/2.jpg)
Outline
•Background on Differential Privacy
• 𝜖-Differential Privacy: Single Dimensional Setting
• 𝜖-Differential Privacy: Multiple Dimensional Setting
• (𝜖, 𝛿)-Differential Privacy
•Conclusion
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 2
![Page 3: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/3.jpg)
•Vast amounts of personal information are collected
•Data analysis produces a lot of useful applications
Motivation
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 3
![Page 4: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/4.jpg)
•How to release the data while protecting individual’s privacy?
Motivation
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 4
Big DataRelease data/info.
Analyst
Applications
![Page 5: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/5.jpg)
Motivation
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 5
• How to protect PRIVACY resilient to attacks with arbitrary side information?
![Page 6: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/6.jpg)
Motivation
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 6
• How to protect PRIVACY resilient to attacks with arbitrary side information?
• Answer: randomized releasing mechanism
![Page 7: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/7.jpg)
Motivation
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 7
• How to protect PRIVACY resilient to attacks with arbitrary side information?
• Answer: randomized releasing mechanism
•How much randomness is needed?• completely random (no utility)
• deterministic (no privacy)
![Page 8: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/8.jpg)
Motivation
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 8
• How to protect PRIVACY resilient to attacks with arbitrary side information?
• Answer: randomized releasing mechanism
•How much randomness is needed?• completely random (no utility)
• deterministic (no privacy)
• Differential Privacy [Dwork et. al. 06]: one way to quantify the level of randomness and privacy
![Page 9: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/9.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9
database
Collect data
Users
Database Curator
Analyst
Query
Released info.
• Database Curator: How to answer a query, providing usefuldata information to the analyst, while still protecting the privacy of each user.
![Page 10: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/10.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 10
database
Collect data
Users
Database Curator
Analyst
Query
Released info.
dataset: D query function: 𝑞 (D) randomized released mechanism: K DAge
A: 20
B: 35
C: 43
D: 30
q: How many people are older than 32?
q(D) = 2
K(D) = 1 w.p. 1/8K(D) = 2 w.p. 1/2K(D) = 3 w.p. 1/4K(D) = 4 w.p. 1/8
![Page 11: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/11.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 11
• Two neighboring datasets: differ only at one element
Age
A: 20
B: 35
C: 43
D: 30
Age
A: 20
B: 35
D: 30
𝐷1 𝐷2
![Page 12: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/12.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 12
• Two neighboring datasets: differ only at one element
Age
A: 20
B: 35
C: 43
D: 30
Age
A: 20
B: 35
D: 30
𝐷1 𝐷2
Differential Privacy: presence or absence of any individual record in the dataset should not affect the released information significantly.
𝐾 𝐷1 ≈ 𝐾(𝐷2)
![Page 13: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/13.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 13
• A randomized mechanism 𝐾 gives 𝝐-differential privacy, if for any two neighboring datasets 𝐷1, 𝐷2 , and all 𝑆 ⊂Range(K),
𝐏𝐫 𝐊 𝐃𝟏 ∈ 𝐒 ≤ 𝐞𝛜 𝐏𝐫 𝐊 𝐃𝟐 ∈ 𝐒
![Page 14: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/14.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 14
• A randomized mechanism 𝐾 gives 𝝐-differential privacy, if for any two neighboring datasets 𝐷1, 𝐷2 , and all 𝑆 ⊂Range(K),
𝐏𝐫 𝐊 𝐃𝟏 ∈ 𝐒 ≤ 𝐞𝛜 𝐏𝐫 𝐊 𝐃𝟐 ∈ 𝐒
Make hypothesis testing hard:𝑒𝜖𝑃𝐹𝐴 + 𝑃𝑀𝐷 ≥ 1𝑃𝐹𝐴 + 𝑒
𝜖𝑃𝑀𝐷 ≥ 1
![Page 15: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/15.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 15
• A randomized mechanism 𝐾 gives 𝝐-differential privacy, if for any two neighboring datasets 𝐷1, 𝐷2 , and all 𝑆 ⊂Range(K),
𝐏𝐫 𝐊 𝐃𝟏 ∈ 𝐒 ≤ 𝐞𝛜 𝐏𝐫 𝐊 𝐃𝟐 ∈ 𝐒
• 𝝐 quantifies the level of privacy• 𝜖 → 0, high privacy• 𝜖 → +∞, low privacy• a social question to choose 𝝐 (can be 0.01, 0.1, 1, 10 …)
![Page 16: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/16.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 16
• Q: How much perturbation needed to achieve 𝜖-DP?
• A: depends on how different 𝑞 𝐷1 , 𝑞(𝐷2) are
![Page 17: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/17.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 17
• Q: How much perturbation needed to achieve 𝜖-DP?
• A: depends on how different 𝑞 𝐷1 , 𝑞(𝐷2) are
𝑞 𝐷1 , 𝑞(𝐷2)
If 𝑞 𝐷1 = 𝑞 𝐷2 , no perturbation is needed.
![Page 18: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/18.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 18
• Q: How much perturbation needed to achieve 𝜖-DP?
• A: depends on how different 𝑞 𝐷1 , 𝑞(𝐷2) are
𝑓𝐷1(𝑥) 𝑓𝐷2(𝑥)
𝑞 𝐷1 𝑞(𝐷2)
If |𝑞 𝐷1 − 𝑞 𝐷2 | is small, use small perturbation
![Page 19: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/19.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 19
• Q: How much perturbation needed to achieve 𝜖-DP?
• A: depends on how different 𝑞 𝐷1 , 𝑞(𝐷2) are
𝑓𝐷1(𝑥)𝑓𝐷2(𝑥)
𝑞 𝐷1 𝑞(𝐷2)
If |𝑞 𝐷1 − 𝑞 𝐷2 | is large, use large perturbation
![Page 20: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/20.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 20
• Global Sensitivity Δ: how different when q is applied to neighboring datasets
𝜟 ≔ 𝐦𝐚𝐱(𝑫𝟏,𝑫𝟐) 𝒂𝒓𝒆 𝒏𝒆𝒊𝒈𝒉𝒃𝒐𝒓𝒔
|𝒒 𝑫𝟏 − 𝒒 𝑫𝟐 |
• Example: for a count query, Δ = 1
![Page 21: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/21.jpg)
Background on Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 21
• Laplace Mechanism:
𝑲 𝑫 = 𝒒 𝑫 + 𝑳𝒂𝒑(Δ
𝜖),
𝐿𝑎𝑝𝛥
𝜖is a r.v. with p.d.f 𝑓 𝑥 =
1
2𝜆𝑒−|𝑥|
𝜆 , 𝜆 =Δ
𝜖
• Basic tool in DP
𝑓𝐷1(𝑥) 𝑓𝐷2(𝑥)
![Page 22: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/22.jpg)
Optimality of Existing work?
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 22
• Laplace Mechanism:
𝑲 𝑫 = 𝒒 𝑫 + 𝑳𝒂𝒑(Δ
𝜖),
• Two Questions:• Is data-independent perturbation optimal?
• Assume data-independent perturbation, is Laplacian distribution optimal?
![Page 23: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/23.jpg)
Optimality of Existing work?
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 23
• Laplace Mechanism:
𝑲 𝑫 = 𝒒 𝑫 + 𝑳𝒂𝒑(Δ
𝜖),
• Two questions:• Is data-independent perturbation optimal?• Assume data-independent perturbation, is Laplacian distribution
optimal?
• Our results:• data-independent perturbation is optimal• Laplacian distribution is not optimal: the optimal is staircase
distribution
![Page 24: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/24.jpg)
Part One:
The Optimal Mechanism in 𝝐-Differential Privacy: Single Dimensional Setting
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 24
presented in my preliminary exam
![Page 25: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/25.jpg)
Recap of Part One: Problem Formulation
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 25
m𝑖𝑛𝑖𝑚𝑖𝑧𝑒 supt∈𝑅 𝑥∈𝑅 𝐿 𝑥 𝜈𝑡(𝑑𝑥)
s.t. 𝛎t1 S ≤ eϵ𝛎t2 S + t1 − t2 ,
∀measurable set S, ∀t1, t2 ∈ R, s. t. t1 − t2 ≤ Δ,
𝑳: 𝑅 → 𝑅 cost function on the noise𝛎𝑡 noise probability distribution given query output tΔ global sensitivity
![Page 26: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/26.jpg)
11/07/2013 26
• Optimality of query-output independent perturbationIn the optimal mechanism, 𝜈𝑡 is independent of 𝑡 (under a technical condition).
• Staircase MechanismOptimal noise probability distribution has staircase-shaped p.d.f.
Recap of Part One: Main Results
Minimize L x P dxs.t. Pr X ∈ S ≤ eϵ Pr X ∈ S + d ,∀ d ≤ Δ,measurable set S
![Page 27: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/27.jpg)
11/07/2013 27
• ℓ𝟏: 𝐋 𝐱 = |𝐱|
• ℓ𝟐: 𝐋 𝒙 = 𝐱𝟐
Recap of Part One: Applications
𝑽 𝑷𝜸∗ = 𝚫𝒆𝝐𝟐
𝒆𝝐 − 𝟏
𝜖 → 0, the additive gap → 0
𝜖 → +∞, 𝑉 𝑃𝛾∗ = Θ(Δe−𝜖
2)
𝑉𝐿𝑎𝑝 =Δ
𝜖
𝜖 → 0, the additive gap ≤ 𝑐Δ2
𝜖 → +∞, 𝑉 𝑃𝛾∗ = Θ(Δ2 e−
2𝜖
3 )
𝑉𝐿𝑎𝑝 =2Δ2
𝜖2
Huge improvement in low privacy regime
![Page 28: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/28.jpg)
11/07/2013 28
• Properties of 𝛾∗ (for 𝐿 𝑥 = 𝑥 𝑚)
• Extension to Discrete Setting
• Extension to Abstract Setting
Recap of Part One: Applications
![Page 29: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/29.jpg)
Conclusion of Part One:
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 29
• Fundamental tradeoff between privacy and utility in 𝜖-Differential Privacy
• Staircase Mechanism, optimal mechanism for single real-valued query• Huge improvement in low privacy regime
• Extension to discrete setting and abstract setting
![Page 30: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/30.jpg)
Part Two:
The Optimal Mechanism in 𝝐-Differential Privacy: Multiple Dimensional Setting
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 30
![Page 31: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/31.jpg)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 31
• Query output can have multiple components
𝒒 𝑫 = 𝒒𝟏 𝑫 , 𝒒𝟐 𝑫 ,… , 𝒒𝒅 𝑫 ∈ 𝑹𝒅
Multiple Dimensional Setting
![Page 32: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/32.jpg)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 32
• Query output can have multiple components
𝒒 𝑫 = 𝒒𝟏 𝑫 , 𝒒𝟐 𝑫 ,… , 𝒒𝒅 𝑫 ∈ 𝑹𝒅
• If all components are uncorrelated, perturb each component independently to preserve DP.
Multiple Dimensional Setting
![Page 33: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/33.jpg)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 33
• Query output can have multiple components
𝒒 𝑫 = 𝒒𝟏 𝑫 , 𝒒𝟐 𝑫 ,… , 𝒒𝒅 𝑫 ∈ 𝑹𝒅
• If all components are uncorrelated, perturb each component independently to preserve DP.
• Composition Theorem:
For each component 𝒒𝑖 (𝑫) preserving 𝜖-DP, end-to-end achieves 𝒅𝝐-DP.
Multiple Dimensional Setting
![Page 34: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/34.jpg)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 34
• Query output can have multiple components
𝒒 𝑫 = 𝒒𝟏 𝑫 , 𝒒𝟐 𝑫 ,… , 𝒒𝒅 𝑫 ∈ 𝑹𝒅
• For some query, all components are coupled together.
Histogram query: one user can affect only one component
Multiple Dimensional Setting
![Page 35: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/35.jpg)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 35
• Query output can have multiple components
𝒒 𝑫 = 𝒒𝟏 𝑫 , 𝒒𝟐 𝑫 ,… , 𝒒𝒅 𝑫 ∈ 𝑹𝒅
• For some query, all components are coupled together.
• Global sensitivity
𝚫 ≔ max𝑫𝟏,𝑫𝟐 𝒂𝒓𝒆 𝒏𝒆𝒊𝒈𝒉𝒃𝒐𝒓𝒔
|𝒒 𝑫𝟏 − 𝒒 𝑫𝟐 | 𝟏
Histogram query: 𝚫 = 1
Multiple Dimensional Setting
![Page 36: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/36.jpg)
Problem Formulation:
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 36
Query-output independent perturbation:
𝑲 𝑫 = 𝒒(𝑫) + 𝑿= (𝒒𝟏 𝑫 + 𝒙𝟏, 𝒒𝟐 𝑫 + 𝒙𝟐, … , 𝒒𝒅 𝑫 + 𝒙𝒅)
![Page 37: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/37.jpg)
Problem Formulation:
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 37
Query-output independent perturbation:
𝑲 𝑫 = 𝒒(𝑫) + 𝑿= (𝒒𝟏 𝑫 + 𝒙𝟏, 𝒒𝟐 𝑫 + 𝒙𝟐, … , 𝒒𝒅 𝑫 + 𝒙𝒅)
Cost function on the noise:
𝐿: 𝑅𝑑 → 𝑅
![Page 38: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/38.jpg)
Optimization problem:
𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑅𝑑𝐿 𝑥1, … , 𝑥𝑑 𝑃(𝑑𝑥1⋯𝑑𝑥𝑑)
s.t. 𝑃 𝑆 ≤ 𝑒𝜖𝑃 𝑆 + 𝒕 , ∀𝑆 ⊂ 𝑅𝑑 , 𝒕 1 ≤ Δ
Problem Formulation:
38
Query-output independent perturbation:
𝑲 𝑫 = 𝒒(𝑫) + 𝑿= (𝒒𝟏 𝑫 + 𝒙𝟏, 𝒒𝟐 𝑫 + 𝒙𝟐, … , 𝒒𝒅 𝑫 + 𝒙𝒅)
Cost function on the noise:
𝐿: 𝑅𝑑 → 𝑅
![Page 39: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/39.jpg)
Our Result: Optimality of Staircase Mechanism
39
Theorem: If 𝐿 𝑥1, … , 𝑥𝑑 = 𝑥1 +⋯+ 𝑥𝑑 , 𝑑 = 2, then optimal probability distribution has multidimensional staircase-shaped p.d.f.
![Page 40: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/40.jpg)
Our Result: Optimality of Staircase Mechanism
40
Theorem: If 𝐿 𝑥1, … , 𝑥𝑑 = 𝑥1 +⋯+ 𝑥𝑑 , 𝑑 = 2, then optimal probability distribution has multidimensional staircase-shaped p.d.f.
![Page 41: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/41.jpg)
Our Result: Optimality of Staircase Mechanism
41
Theorem: If 𝐿 𝑥1, … , 𝑥𝑑 = 𝑥1 +⋯+ 𝑥𝑑 , 𝑑 = 2, then optimal probability distribution has multidimensional staircase-shaped p.d.f.
Conjecture: the result holds for any 𝒅
![Page 42: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/42.jpg)
Proof Ideas
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 42
Step 1: Argue only need to consider multi-dimensional piecewise constant p.d.f. by averaging
𝑥1
𝑥2
0
Averaging preserves DP
![Page 43: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/43.jpg)
Proof Ideas
PhD Final Exam of Quan Geng, ECE, UIUC 43
Step 1: Argue only need to consider multi-dimensional piecewise constant p.d.f. by averaging
𝑥1
𝑥2
Averaging preserves DP𝑃1 ≤ 𝑒𝜖 𝑃5𝑃1 ≤ 𝑒𝜖 𝑃6𝑃1 ≤ 𝑒𝜖 𝑃16𝑃1 ≤ 𝑒𝜖 𝑃7𝑃1 ≤ 𝑒𝜖 𝑃15……
𝑃4 ≤ 𝑒𝜖 𝑃12𝑃4 ≤ 𝑒𝜖 𝑃13𝑃4 ≤ 𝑒𝜖 𝑃14𝑃4 ≤ 𝑒𝜖 𝑃15𝑃4 ≤ 𝑒𝜖 𝑃16
𝑖=14 𝑃𝑖
4≤ 𝑒𝜖 𝑖=516 𝑃𝑖
12
![Page 44: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/44.jpg)
Proof Ideas
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 44
𝑥2
0
Averaging preserves DP
Formulate a matrix filling-in problem• row sum is a constant• column sum is a constant
Explicit formula for 𝒅 = 𝟐, and arbitrary 𝚫
![Page 45: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/45.jpg)
Proof Ideas
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 45
Due to Step 1, the p.d.f. is a function of 𝑿 𝟏
Step 2: Monotonicity
Step 3: Geometrically Decaying
Step 4: Staircase-Shape
![Page 46: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/46.jpg)
Optimal 𝜸∗ for ℓ𝟏 cost function (d=2)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 46
![Page 47: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/47.jpg)
Asymptotic analysis
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 47
Laplacian Mechanism: 𝑽 =𝟐𝚫
𝝐
Multidimensional Staircase Mechanism:High privacy regime(𝜖 → 0):
Low privacy regime(𝜖 → ∞):
![Page 48: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/48.jpg)
Asymptotic analysis
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 48
Laplacian Mechanism: 𝑽 =𝟐𝚫
𝝐
Multidimensional Staircase Mechanism:High privacy regime(𝜖 → 0):
Low privacy regime(𝜖 → ∞):Independent Staircase Mechanism:
𝑉 = Θ(2Δ𝑒−𝜖4)
![Page 49: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/49.jpg)
Conclusion of Part Two:
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 49
• Extension of Optimality of Staircase Mechanism to Multi-dimensional Setting (𝑑 = 2)• Conjecture: holds for any dimension
•Approximate Optimality of Laplacian Mechanism in High Privacy Regime
•Huge Improvement in Low Privacy Regime
![Page 50: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/50.jpg)
Part Three:
The Optimal Mechanism in (𝝐, 𝜹)-Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 50
![Page 51: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/51.jpg)
Part Three: (𝜖,𝛿)-differential privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 51
• (𝜖, 𝛿)-differential privacy
𝐏𝐫 𝐊 𝐃𝟏 ∈ 𝐒 ≤ 𝐞𝛜 𝐏𝐫 𝐊 𝐃𝟐 ∈ 𝐒 + 𝛅
• Two special cases:• (𝝐, 𝟎)-differential privacy
• well studied in Part 1 and Part 2
• (𝟎, 𝜹)-differential privacy
![Page 52: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/52.jpg)
52
• (𝜖, 𝛿)-differential privacy
𝐏𝐫 𝐊 𝐃𝟏 ∈ 𝐒 ≤ 𝐞𝛜 𝐏𝐫 𝐊 𝐃𝟐 ∈ 𝐒 + 𝛅
• Make hypothesis testing hard:
𝑒𝜖𝑃𝐹𝐴 + 𝑃𝑀𝐷 ≥ 1 − 𝛿
𝑃𝐹𝐴 + 𝑒𝜖𝑃𝑀𝐷 ≥ 1 − 𝛿
(𝜖,𝛿)-Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
![Page 53: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/53.jpg)
53
• (𝜖, 𝛿)-differential privacy
𝐏𝐫 𝐊 𝐃𝟏 ∈ 𝐒 ≤ 𝐞𝛜 𝐏𝐫 𝐊 𝐃𝟐 ∈ 𝐒 + 𝛅
• Make hypothesis testing hard:
𝑒𝜖𝑃𝐹𝐴 + 𝑃𝑀𝐷 ≥ 1 − 𝛿
𝑃𝐹𝐴 + 𝑒𝜖𝑃𝑀𝐷 ≥ 1 − 𝛿
• Standard approach: adding Gaussian noise
(𝜖,𝛿)-Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
![Page 54: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/54.jpg)
54
• (𝜖, 𝛿)-differential privacy 𝐏𝐫 𝐊 𝐃𝟏 ∈ 𝐒 ≤ 𝐞
𝛜 𝐏𝐫 𝐊 𝐃𝟐 ∈ 𝐒 + 𝛅
• What is the optimal noise probability distribution in this setting?
• Our Results:• (0, 𝛿)-DP
Optimality of Uniform Noise Mechanism
• (𝜖, 𝛿)-DPNot much more general than (𝜖, 0)- and (0, 𝛿)-DP Near-Optimality of Laplacian and Uniform Noise Mechanism in high privacy regime as 𝜖, 𝛿 → 0,0
(𝜖,𝛿)-Differential Privacy
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
![Page 55: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/55.jpg)
55
Problem Formulation
V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ eϵP S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
![Page 56: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/56.jpg)
56
V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ P S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
(0,𝛿)-Differential Privacy
• Theorem:
If Δ = 1, assuming 1
2𝛿is an integer, optimal noise P.D. is
𝛿
𝑖
P(𝑖)
−1
2𝛿
1
2𝛿− 1
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
![Page 57: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/57.jpg)
57
V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ P S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
(0,𝛿)-Differential Privacy
• Theorem:
If Δ = 1, assuming 1
2𝛿is an integer, optimal noise P.D. is
𝛿
𝑖
P(𝑖)
−1
2𝛿
1
2𝛿− 1
Proof idea: choose 𝑆 = 𝑆𝑘 = {𝑙: 𝑙 ≥ 𝑘},then 𝑃 𝑘 ≤ 𝛿, ∀𝑘
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
![Page 58: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/58.jpg)
58
V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ P S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
(0,𝛿)-DP: General 𝚫
• Upper Bound: Uniform Noise Mechanism
𝛿
Δ
𝑖
P(𝑖)
−Δ
2𝛿
Δ
2𝛿− 1
𝑉∗ ≤ 𝑉𝑈𝐵 ≔ 2 𝑖=1
𝛿2Δ−1 𝛿
Δ𝐿 𝑖 +
𝛿
Δ𝐿(Δ
2𝛿)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
![Page 59: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/59.jpg)
59
V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ P S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
(0,𝛿)-DP: General 𝚫
• Lower Bound: Duality of Linear Programming
choose 𝑆 = 𝑆𝑘 = {𝑙: 𝑙 ≥ 𝑘}, then 𝑖=𝑘𝑘+Δ−1𝑃(𝑖) ≤ 𝛿, ∀𝑘
![Page 60: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/60.jpg)
60
V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ P S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
(0,𝛿)-DP: General 𝚫
• Lower Bound: Duality of Linear Programming
choose 𝑆 = 𝑆𝑘 = {𝑙: 𝑙 ≥ 𝑘}, then 𝑖=𝑘𝑘+Δ−1𝑃(𝑖) ≤ 𝛿, ∀𝑘
𝑉𝐿𝐵 = 2𝛿
𝑖=0
12𝛿−1
𝐿(1 + 𝑖Δ)
![Page 61: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/61.jpg)
61
(0,𝛿)-DP: Comparison of 𝑽𝑳𝑩, 𝑽𝑼𝑩
• 𝑳(𝒊) = |𝒊|
11/07/2013
Constant additive gap
![Page 62: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/62.jpg)
62
(0,𝛿)-DP: Comparison of 𝑽𝑳𝑩, 𝑽𝑼𝑩
• 𝑳(𝒊) = |𝒊|
• 𝑳(𝒊) = 𝒊 𝟐
11/07/2013
Constant additive gap
Constant multiplicative gap
![Page 63: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/63.jpg)
63
(0,𝛿)-DP: Comparison of 𝑽𝑳𝑩, 𝑽𝑼𝑩
• 𝑳(𝒊) = |𝒊|
• 𝑳(𝒊) = 𝒊 𝟐
• 𝑳(𝒊) = 𝒊 𝒎
11/07/2013
Constant additive gap
Constant multiplicative gap
![Page 64: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/64.jpg)
64
(𝝐,𝛿)-DP: Upper Bound
•Both (𝝐,0)-DP and (0,𝛿)-DP imply (𝝐,𝛿)-DP
𝑽∗ ≤ 𝒎𝒊𝒏 (𝑽𝑳𝒂𝒑, 𝑽𝑼𝒏𝒊𝒇𝒐𝒓𝒎)
V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ e𝜖P S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
Laplacian Mechanism:
continuous: 𝑓 𝑥 = 12𝜆𝑒−|𝑥|
𝜆 , 𝜆 =Δ
𝜖
discrete: 𝑃 𝑘 = 1−𝑒−𝜖Δ
1+𝑒−𝜖Δ
𝑒−|𝑘|𝜖
Δ
![Page 65: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/65.jpg)
65
(𝝐,𝛿)-DP: Lower Bound V∗ ≔ m𝑖𝑛
i=−∞
+∞
L i P(i)
s.t. P S ≤ e𝜖P S + 𝑑 + 𝛿,∀S ⊂ 𝑍, 𝑑 ∈ 𝑍, 𝑑 ≤ Δ
• Lower Bound: Duality of Linear Programming
choose 𝑆 = 𝑆𝑘 = {𝑙: 𝑙 ≥ 𝑘}, by symmetry
![Page 66: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/66.jpg)
66
(𝝐,𝛿)-DP: Comparison of 𝑽𝑳𝑩, 𝑽𝑼𝑩
• 𝑳(𝒊) = |𝒊|, as 𝜖, 𝛿 → (0,0)
𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
5.29≤ 𝑉𝐿𝐵 ≤ 𝑉
∗ ≤ 𝑉𝑈𝐵 = 𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
= Θ(min1
𝜖,1
𝛿)
![Page 67: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/67.jpg)
67
(𝝐,𝛿)-DP: Comparison of 𝑽𝑳𝑩, 𝑽𝑼𝑩
• 𝑳(𝒊) = |𝒊|, as 𝜖, 𝛿 → (0,0)
𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
5.29≤ 𝑉𝐿𝐵 ≤ 𝑉
∗ ≤ 𝑉𝑈𝐵 = 𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
• 𝑳 𝒊 = 𝒊 𝟐, as 𝜖, 𝛿 → (0,0)
𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
40≤ 𝑉𝐿𝐵 ≤ 𝑉
∗ ≤ 𝑉𝑈𝐵 = 𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
= Θ(min1
𝜖,1
𝛿)
= Θ(𝑚𝑖𝑛1
𝜖2,1
𝛿2)
![Page 68: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/68.jpg)
68
(𝝐,𝛿)-DP: Multi-Dimensional Setting
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
V∗ ≔ m𝑖𝑛
𝑍𝑑
L i1, 𝑖2, … , 𝑖𝑑 P(i1, 𝑖2, … , 𝑖𝑑)
s.t. P S ≤ P S + 𝑑 + 𝛿,∀S ⊂ 𝑍𝑑 , 𝑑 ∈ 𝑍𝑑 , 𝑑 1 ≤ Δ• Upper Bound
Uniform Noise Mechanism
Discrete Laplacian Mechanism
𝑃 𝑖1, ⋯ , 𝑖𝑑 =1 − 𝜆
1 + 𝜆
𝑑
𝜆 𝑖1 +⋯+|𝑖𝑑|, 𝜆 = 𝑒−𝜖/Δ
![Page 69: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/69.jpg)
69
(𝝐,𝛿)-DP: Multi-Dimensional Setting
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
V∗ ≔ m𝑖𝑛
𝑍𝑑
L i1, 𝑖2, … , 𝑖𝑑 P(i1, 𝑖2, … , 𝑖𝑑)
s.t. P S ≤ P S + 𝑑 + 𝛿,∀S ⊂ 𝑍𝑑 , 𝑑 ∈ 𝑍𝑑 , 𝑑 1 ≤ Δ
• Lower Bound1. Choose certain sets 𝑆 = 𝑆𝑘
𝑚 ≔ { 𝑖1, ⋯ , 𝑖𝑑 ∈ 𝑍𝑑|𝑖𝑚 ≥ 𝑘}
2. Write down Linear program, and derive the dual problem3. Find proper dual variables to get lower bound
![Page 70: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/70.jpg)
70
(𝝐,𝛿)-DP: Multi-Dimensional Setting
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
Primal Problem Dual Problem
![Page 71: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/71.jpg)
71
(𝝐,𝛿)-DP: Multi-Dimensional Setting
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC
• (𝟎,𝛿)-DP:
Uniform Noise Mechanism is Optimal when 𝜟 = 𝟏
ℓ1 ℓ2
• (𝝐,𝛿)-DP:
𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
𝐶≤ 𝑉𝐿𝐵 ≤ 𝑉
∗ ≤ 𝑉𝑈𝐵 = 𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
𝐶 ≈ 9.49, 𝑓𝑜𝑟 ℓ1
𝐶 ≈ 113, 𝑓𝑜𝑟 ℓ2
![Page 72: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/72.jpg)
Conclusion of Part Three
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 72
•Near-Optimality of Uniform Noise Mechanism in 0, 𝛿 -DP
• (𝜖, 𝛿)-DP is not much more general than (0, 𝛿)-DP and (𝜖,0)-DP
𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
𝐶≤ 𝑉𝐿𝐵 ≤ 𝑉
∗ ≤ 𝑉𝑈𝐵 = 𝑚𝑖𝑛 (𝑉𝐿𝑎𝑝, 𝑉𝑈𝑛𝑖𝑓𝑜𝑟𝑚)
![Page 73: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/73.jpg)
Conclusion
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 73
• Fundamental tradeoff between privacy and utility in Differential Privacy
• Staircase Mechanism, optimal mechanism in 𝜖-DP• Huge improvement in low privacy regime• Extension to Multi-Dimensional Setting
• Uniform Noise Mechanism, near-optimal mechanism in (0, 𝛿)-DP
• (𝜖, 𝛿)-DP is not much more general than (0, 𝛿)-DP and (𝜖,0)-DP
![Page 74: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/74.jpg)
Acknowledgment
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 74
Thanks to my advisor, Prof. Pramod Viswanath
![Page 75: The Optimal Mechanism in Differential Privacy · 11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 9 database Collect data Users Database Curator Analyst Query Released info. •Database](https://reader033.vdocuments.us/reader033/viewer/2022050110/5f477ba50380c3418c66fe03/html5/thumbnails/75.jpg)
Thank you!
11/07/2013 PhD Final Exam of Quan Geng, ECE, UIUC 75