Download - The Sparse Vector Technique
![Page 1: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/1.jpg)
Lecture 12 : 590.03 Fall 12 1
The Sparse Vector Technique
CompSci 590.03Instructor: Ashwin Machanavajjhala
![Page 2: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/2.jpg)
Lecture 12 : 590.03 Fall 12 2
Announcement
• Project proposal submission deadline is Fri, Oct 12 noon.
• How to write the proposal?– Just like any paper …– … Abstract, Introduction, Notation, Problem Statement, Related Work – Instead of algorithms and results sections, you will have section describing
how you will solve the problem.
![Page 3: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/3.jpg)
Lecture 12 : 590.03 Fall 12 3
Recap: Laplace MechanismThm: If sensitivity of the query is S, then adding Laplace noise with
parameter λ guarantees ε-differential privacy, when
λ = S/ε
Sensitivity: Smallest number s.t. for any d, d’ differing in one entry, || q(d) – q(d’) || ≤ S(q)
Histogram query: Sensitivity = 2• Variance / error on each entry = 2λ2 = 2x4/ε2
![Page 4: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/4.jpg)
Lecture 12 : 590.03 Fall 12 4
Cohort Size Estimation Problem
Population of medical patients
Are there at least 200 individuals who are male
cancer survivors, between 20-30, who were admitted
for surgery
Are there at least 200 male cancer survivors who are
between ages of 20 and 30
Are there at least 200 individuals who are male
cancer survivors and admitted for surgery
![Page 5: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/5.jpg)
Lecture 12 : 590.03 Fall 12 5
Cohort Size Estimation Problem• A set of queries {Q1, Q2, Q3, …, Qn}
• Each query Qi : Number of tuples satisfying property pi > τ ?– If answer is yes, return the number of tuples satisfying that property
And, Researcher performs additional analysis– If answer is no, then return NULL.
• Sensitivity of each Qi = ?
• How do we answer using differential privacy?
= 1
![Page 6: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/6.jpg)
Lecture 12 : 590.03 Fall 12 6
Cohort Size Estimation ProblemLaplace mechanism:
• Sensitivity of all queries is: n
• For each query: qi’ = Qi(D) + Lap(n/ε)• Return qi’ if qi’ > τ
Return φ if qi’ < τ
![Page 7: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/7.jpg)
Lecture 12 : 590.03 Fall 12 7
Accuracy• We will say that an algorithm is (α, β)-accurate if for a sequence
of queries Q1, Q2, …, Qn if with probability > 1-β, the following holds:
|qi’ – Qi(D)| < α if qi’ ≠ φ Qi(D) < T + α if qi’ = φ
![Page 8: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/8.jpg)
Lecture 12 : 590.03 Fall 12 8
Accuracy of Laplace Mechanism
![Page 9: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/9.jpg)
Lecture 12 : 590.03 Fall 12 9
Cohort Estimation Problem• In many exploratory situations, only a small number c of the
queries actually have a count > τ
• However, accuracy depends on the total number of queries, not just the queries that cross the threshold, – Even though we do not return an answer otherwise.
• Is there a mechanism where you need to pay when the count is > τ ?
![Page 10: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/10.jpg)
Lecture 12 : 590.03 Fall 12 10
Sparse Vector Technique
• Set count = 0• Set τ’ = τ + Lap(2/ε)• For each query: qi’ = Qi(D) + Lap(2c/ε)• If qi’ ≥ τ’ & count < c,
count++Return qi’
Else if qi’ < τ’Return φ
Else // count ≥ cAbort
Instead of Lap(n/ε)
Answer at most c queries positively
Use a noisy threshold
![Page 11: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/11.jpg)
Lecture 12 : 590.03 Fall 12 11
Sparse Vector Technique: PrivacyPrevious answers (current answer is not
independent of previous answers)
![Page 12: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/12.jpg)
Lecture 12 : 590.03 Fall 12 12
Sparse Vector Technique: Privacy
At most c queries answered positively
Independent of the number of queries answered with NULL
![Page 13: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/13.jpg)
Lecture 12 : 590.03 Fall 12 13
Sparse Vector Technique: Privacy• Let AZ(D) be the set of noise values { vi = qi’ – Qi(D) }
that result in the observed φ answers when τ’ = Z.
• If we changed D to D’,
• If Qi(D) + vi < Z, then Qi(D’) + vi ≤ Qi(D) + 1 + vi ≤ Z+1
• If Qi(D’) + vi < Z-1, then Qi(D) + vi ≤ Qi(D’) + 1 + vi ≤ Z
![Page 14: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/14.jpg)
Lecture 12 : 590.03 Fall 12 14
Sparse Vector Technique: Privacy• Let AZ(D) be the set of noise values { vi = qi’ – Qi(D) }
that result in the observed φ answers when τ’ = Z.
• If we changed D to D’,
• Also, from Laplace mechanism,
![Page 15: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/15.jpg)
Lecture 12 : 590.03 Fall 12 15
Sparse Vector Technique: Privacy
![Page 16: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/16.jpg)
Lecture 12 : 590.03 Fall 12 16
Sparse Vector Technique: Privacy
• Pay c ∙ ε1 (=ε/2) privacy for the questions that have a count greater than the noisy threshold.
• You pay ε2 (=ε/2) privacy for adding noise to the threshold.
• All the questions whose counts are lower than the threshold are answered for free!
![Page 17: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/17.jpg)
Lecture 12 : 590.03 Fall 12 17
Sparse Vector Technique: AccuracyTheorem: For any queries Q1, Q2, …, Qk such that
|{ i : Qi(D) > τ - α}| ≤ cThen, the sparse vector technique:1. does not abort, and
2. is (α,β)-accurate for
Recall: Laplace mechanism is (α,β)-accurate for
![Page 18: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/18.jpg)
Lecture 12 : 590.03 Fall 12 18
Accuracy• We will say that an algorithm is (α, β)-accurate if for a sequence
of queries Q1, Q2, …, Qn if with probability > 1-β the algorithm does not abort and the following holds:
|qi’ – Qi(D)| < α if qi’ ≠ φ, or qi’ ≥ τ’
Qi(D) < T + α if qi’ = φ, or qi’ < τ’
![Page 19: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/19.jpg)
Lecture 12 : 590.03 Fall 12 19
Sparse Vector Technique: Accuracy• Suppose
• When qi’ ≠ φ,
• When qi’ = φ, then qi’ < τ’
• And, the algorithm always aborts:
![Page 20: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/20.jpg)
Lecture 12 : 590.03 Fall 12 20
Sparse Vector Technique: Accuracy
![Page 21: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/21.jpg)
Lecture 12 : 590.03 Fall 12 21
Summary of Sparse Vector Technique• If you have many low sensitivity queries, and you only expect a
few of the queries to be useful.
• Sparse vector techniques allows you to pay only for the positively answered queries.
• Much smaller error than the Laplace mechanism.
![Page 22: The Sparse Vector Technique](https://reader034.vdocuments.us/reader034/viewer/2022051401/56813be2550346895da50e70/html5/thumbnails/22.jpg)
Lecture 12 : 590.03 Fall 12 22
Next Class• Multiplicative Weights Algorithms
– General paradigm for algorithm design
– Application to privately answering queries– Application to privately publishing a dataset