bayesian inferencetarmo/tsem15/gaboardi0312-slides.pdf · marco gaboardi university of dundee...
TRANSCRIPT
![Page 1: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/1.jpg)
Marco GaboardiUniversity of Dundee
Verifying Differentially Private Bayesian Inference
Joint work with G. Barthe, G.P. Farina, E.J. Gallego Arias, A.Gordon,…
![Page 2: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/2.jpg)
Differentially Private vs Probabilistic Inference
DifferentialPrivacy
ProbabilisticInference
![Page 3: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/3.jpg)
Differentially Private vs Probabilistic Inference
DifferentialPrivacy
ProbabilisticInference
The goal in machine learning is very often similar to the goal in private data analysis. The learner typically wishes to learn some simple rule that explains a data set. However, she wishes this rule to generalize […] Generally, this means that she wants to learn a rule that captures distributional information about the data set on hand, in a way that does not depend too specifically on any single data point.
C. Dwork & A. Roth - The algorithmic Foundations of Differential privacy
![Page 4: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/4.jpg)
Outline
• Bayesian learning
• A language for bayesian learning
• Differential Privacy
• Combining differential privacy and bayesian learning
![Page 5: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/5.jpg)
Bayesian Inference
![Page 6: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/6.jpg)
Probabilistic Inference
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Fun/TabularData
![Page 7: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/7.jpg)
Bayes theorem
Bayes’ theorem
� � � � � �� �
P | PP |
Px y y
y xx
We have just (perhaps without realising it) derived and used Bayes’ theorem to solve a problem.
It inverts a conditional.It follows directly from the product rule.
AC50001 - Introduction to machine learning and data mining - Marco Gaboardi - 2014
� � � � � � � � � �P , P | P P | Px y x y y y x x
![Page 8: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/8.jpg)
Bayes theorem
Bayes’ theorem
� � � � � �� �
P | PP |
Px y y
y xx
We have just (perhaps without realising it) derived and used Bayes’ theorem to solve a problem.
It inverts a conditional.It follows directly from the product rule.
AC50001 - Introduction to machine learning and data mining - Marco Gaboardi - 2014
� � � � � � � � � �P , P | P P | Px y x y y y x x
PriorPosterior
NormalizationFactor
Model
![Page 9: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/9.jpg)
A simple example: bias of a coin
ID disease1 02 13 04 15 16 17 08 19 …
![Page 10: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/10.jpg)
Bayes theorem
Bayes’ theorem
� � � � � �� �
P | PP |
Px y y
y xx
We have just (perhaps without realising it) derived and used Bayes’ theorem to solve a problem.
It inverts a conditional.It follows directly from the product rule.
AC50001 - Introduction to machine learning and data mining - Marco Gaboardi - 2014
� � � � � � � � � �P , P | P P | Px y x y y y x x
PriorPosterior Model
![Page 11: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/11.jpg)
Model/Likelihood: Bernoulli
y = (0,1,0,1,1,1,…,0)Observed values
![Page 12: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/12.jpg)
Model/Likelihood: Bernoulli
The probability mass function f(s,Θ) is
where Θ is the “bias of the coin”. This can also be expressed as:
f(s,Θ) = Θ if s = 1{ 1 - Θ if s = 0
f(s,Θ) = Θs(1-Θ)1-s
y = (0,1,0,1,1,1,…,0)Observed values
![Page 13: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/13.jpg)
Prior: Beta distribution
f(a,b,Θ) = Θa(1-Θ)b
Probability Distribution Function
B(a,b)
Conjugate to the Bernoulli distribution
![Page 14: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/14.jpg)
Prior: Beta distribution
![Page 15: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/15.jpg)
Bias of a Coin
Beta(2500,500)Beta(1500,1500)
![Page 16: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/16.jpg)
Another example
ID Cholesterol1 4.662 5.013 3.45 4 4.12 5 6.35 6 4.45 7 6.068 4.98 9 …
Known variance
![Page 17: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/17.jpg)
Another example
ID Y X1 4.2 1.5 2 2.5 0.2 3 2.2 24 1.1 6.5 5 1.5 8.26 0.5 5.6 7 2.0 2.3 8 0.8 4.3 9 … …
![Page 18: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/18.jpg)
Graphical ModelsID
disease1 02 13 04 15 16 17 08 19 …
![Page 19: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/19.jpg)
Probabilistic PCF
![Page 20: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/20.jpg)
Probabilistic Semantics
![Page 21: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/21.jpg)
Observe for Conditional Distributions
observe z ⇒ X(z) in Y
![Page 22: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/22.jpg)
Observe for Conditional Distributions
observe z ⇒ X(z) in Y Bayes’ theorem
� � � � � �� �
P | PP |
Px y y
y xx
We have just (perhaps without realising it) derived and used Bayes’ theorem to solve a problem.
It inverts a conditional.It follows directly from the product rule.
AC50001 - Introduction to machine learning and data mining - Marco Gaboardi - 2014
� � � � � � � � � �P , P | P P | Px y x y y y x x
![Page 23: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/23.jpg)
Semantics of Observe
![Page 24: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/24.jpg)
Semantics of Observe
Filter and rescaling
![Page 25: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/25.jpg)
Semantics of Observe
Filter and rescaling
![Page 26: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/26.jpg)
Differential Privacy
![Page 27: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/27.jpg)
Private Queries?
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
![Page 28: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/28.jpg)
Private Queries?
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
medicalcorrelation?
queryanswer
![Page 29: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/29.jpg)
Private Queries?
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
answer
query
Does Critias have
cancer?
![Page 30: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/30.jpg)
Private Queries?
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
answer
query
Does Critias have
cancer?
I know he visited Atlantis
From Plato’s Timaeus dialogue
![Page 31: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/31.jpg)
Private Queries?
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Does Critias have
cancer?
I know he visited Atlantis
From Plato’s Timaeus dialogue
![Page 32: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/32.jpg)
Differential Privacy
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 33: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/33.jpg)
Differential Privacy
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
query
answer + noise
medicalcorrelation?
![Page 34: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/34.jpg)
Differential Privacy
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
answer + noise
query
?!?
![Page 35: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/35.jpg)
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: motivation
A. Haeberlen
Motivation: Protecting privacy
2USENIX Security (August 12, 2011)
19144 02-15-1964 flue
19146 05-22-1955 brain tumor
34505 11-01-1988 depression
25012 03-12-1972 diabets
16544 06-14-1956 anemia
...
Fair insurance
plan?
Resend
policy?
DataI know Bob is
born 05-22-1955
in Philadelphia…
A first approach: anonymization.I Using different correlated anonymized data sets one
can learn private data.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
?!?
Differential Privacy
![Page 36: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/36.jpg)
(ε,δ)-Differential Privacy
Definition Given ε,δ ≥ 0, a probabilistic query Q: db → R is (ε,δ)-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S] + δ
![Page 37: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/37.jpg)
(ε,δ)-Differential Privacy
Definition Given ε,δ ≥ 0, a probabilistic query Q: db → R is (ε,δ)-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S] + δ
A query returning a probability distribution
![Page 38: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/38.jpg)
(ε,δ)-Differential Privacy
Definition Given ε,δ ≥ 0, a probabilistic query Q: db → R is (ε,δ)-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S] + δ
Privacy parameters
![Page 39: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/39.jpg)
(ε,δ)-Differential Privacy
Definition Given ε,δ ≥ 0, a probabilistic query Q: db → R is (ε,δ)-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S] + δ
a quantification over all the databases
![Page 40: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/40.jpg)
(ε,δ)-Differential Privacy
Definition Given ε,δ ≥ 0, a probabilistic query Q: db → R is (ε,δ)-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S] + δ
a notion of adjacency or distance
![Page 41: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/41.jpg)
(ε,δ)-Differential Privacy
Definition Given ε,δ ≥ 0, a probabilistic query Q: db → R is (ε,δ)-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S] + δ
and over all the possible outcomes
![Page 42: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/42.jpg)
ε-Differential PrivacyDefinition Given ε ≥ 0, a probabilistic query Q: db → R is ε-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S]
![Page 43: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/43.jpg)
Let’s substitute a concrete instance:
Let’s use the two quantifiers:exp(-ε)· Pr[Q(b)∈ S] ≤ Pr[Q(b∪{x})∈ S] ≤ exp(ε)· Pr[Q(b)∈ S]
Definition Given ε ≥ 0, a probabilistic query Q: db → R is ε-differentially private iff ∀b1, b2:db differing in one row and for every S⊆R:
Pr[Q(b1)∈ S] ≤ exp(ε)· Pr[Q(b2)∈ S]
Pr[Q(b∪{x})∈ S] ≤ exp(ε)· Pr[Q(b)∈ S]
ε-Differential Privacy
Pr[Q(b∪{x})∈S] Pr[Q(b)∈S]log ≤ε
and so:
![Page 44: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/44.jpg)
Q : db => R probabilistic
Q(b) Q(b∪{x})
ε-Differential Privacy
![Page 45: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/45.jpg)
ε-Differential PrivacyPr[Q(b∪{x})∈S]
Pr[Q(b) ∈ S]log ≤ε
![Page 46: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/46.jpg)
Probability of a bad event∫BQ(b∪{x}) ∫BQ(b)
log ≤ε
![Page 47: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/47.jpg)
Probability of a bad eventDataset bad
eventwithout my data 16%
with my data 16.3%
![Page 48: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/48.jpg)
Pr[Q(b’)∈S] ≦ eε Pr[Q(b) ∈ S] + δ
(ε,δ)-Differential privacy
for b ~1 b’
![Page 49: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/49.jpg)
Pr[Q(b’)∈S] ≦ eε Pr[Q(b) ∈ S] + δ
(ε,δ)-Differential privacy
for b ~1 b’
Program
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Noise
+Differentially Private Program
![Page 50: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/50.jpg)
PP Result
Sensitivity
PP Result’
+
![Page 51: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/51.jpg)
PP Result
Sensitivity
PP Result’
+
Sensitivity
≤K
![Page 52: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/52.jpg)
Calibrating the Noise on the Sensitivity
Program
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Noise( )
+k sensitive
ε-Differentially Private Program
εk
![Page 53: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/53.jpg)
• Relational Refinement Types
• Higher Order Refinements
• Partiality Monad
• Semantic Subtyping
• Approximate Equivalence for Distributions
Components
HOARe2: Higher Order Approximate Relational Refinement Types for DP
Barthe et al,POPL’15
![Page 54: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/54.jpg)
HOARe2: Higher Order Approximate Relational Refinement Types for DP
![Page 55: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/55.jpg)
• Relational Refinement Types
• Higher Order Refinements
• Partiality Monad
• Semantic Subtyping
• Approximate Equivalence for Distributions
Components
Barthe et al,POPL’15
reasoning about two runs of a program
HOARe2: Higher Order Approximate Relational Refinement Types for DP
![Page 56: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/56.jpg)
Program Logic
Program PX Y
![Page 57: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/57.jpg)
Program P
Precondition Postcondition
X Y
Program Logic
![Page 58: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/58.jpg)
Program P
Precondition Postcondition
X Y
x ≥ 0 y ≥ 1
Program Logic
exp
![Page 59: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/59.jpg)
Program P Precondition Postcondition→
Refinement Type
P : {x | Pre(x)} → {y | Post(y)}
![Page 60: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/60.jpg)
LogicalPredicates
Program P Precondition Postcondition→
Refinement Type
P : {x | Pre(x)} → {y | Post(y)}
![Page 61: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/61.jpg)
Program P Precondition Postcondition→
exp : {x | x ≥ 0} → {y | y ≥ 1}Example
Refinement Type
P : {x | Pre(x)} → {y | Post(y)}
![Page 62: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/62.jpg)
Program P
Precondition Postcondition
X1 Y1
Program PX2 Y2
Differential privacy as a Relational Property
![Page 63: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/63.jpg)
Program P
Precondition Postcondition
X1 Y1
Program PX2 Y2
X1 and X2 differs for the presence or
absence of an individual
Pr[Y1∈ S] ≤ exp(ε)· Pr[Y2∈ S]
Differential privacy as a Relational Property
![Page 64: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/64.jpg)
P : {x | Pre(x1,x2)} → {y | Post(y1,y2)}
Program P Precondition Postcondition→
HOARe2:
Relational Refinement Types
![Page 65: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/65.jpg)
P : {x | Pre(x1,x2)} → {y | Post(y1,y2)}
Program P Precondition Postcondition→
HOARe2:
Relational Refinement Types
LogicalRelations
![Page 66: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/66.jpg)
P : {x | Pre(x1,x2)} → {y | Post(y1,y2)}
Program P Precondition Postcondition→
HOARe2:
Relational Refinement Types
exp : {x | x1 ≤ x2 } → {y | y1 ≤ y2}Example: Monotonicity of exponential
![Page 67: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/67.jpg)
• Relational Refinement Types
• Higher Order Refinements
• Partiality Monad
• Semantic Subtyping
• Approximate Equivalence for Distributions
Components
Barthe et al,POPL’15
reasoning about DP
HOARe2: Higher Order Approximate Relational Refinement Types for DP
![Page 68: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/68.jpg)
Lifting of P
P(x1,x2) P*(x1,x2)
Relation over distributionsRelation
![Page 69: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/69.jpg)
iff it exists a dist. μ over AxB s.t.
• μ(x) > 0 implies x∈R
• 𝞹1μ ≦ μ1 and 𝞹2μ ≦ μ2
Lifting of P
• Given dist. μ1 over A and μ2 over B:
μ1 P* μ2
![Page 70: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/70.jpg)
iff it exists a dist. μ over AxB s.t.
• μ(x) > 0 implies x∈R
• 𝞹1μ ≦ μ1 and 𝞹2μ ≦ μ2
• maxA(𝞹iμ(A) - eε μi(A), μi(A) - eε 𝞹iμ(A)) ≦ δ
(ε,δ)-Lifting of P
• Given dist. μ1 over A and μ2 over B:
μ1 P* μ2εδ
![Page 71: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/71.jpg)
Verifying Differential Privacy
C:{x:db|d(y1,y2)≦1}!{y:O|y1 =* y2}
If we can conclude
then C is (ε,δ)-differentially private.
εδ
![Page 72: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/72.jpg)
Programming Languages forDifferentially Private Probabilistic Inference
DifferentialPrivacy
ProbabilisticInference
![Page 73: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/73.jpg)
Programming Languages forDifferentially Private Probabilistic Inference
DifferentialPrivacy
ProbabilisticInference
ProgrammingLanguage
Tools
![Page 74: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/74.jpg)
Adding noise
![Page 75: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/75.jpg)
Adding noise
Program
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Noise
+Differentially Private Program
Probabilistic
![Page 76: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/76.jpg)
Adding noise
![Page 77: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/77.jpg)
Adding noise
![Page 78: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/78.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
![Page 79: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/79.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 80: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/80.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 81: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/81.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noiseDifferential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 82: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/82.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 83: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/83.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
![Page 84: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/84.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
ProbabilisticInference
![Page 85: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/85.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
ProbabilisticInference
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 86: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/86.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
ProbabilisticInference
![Page 87: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/87.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
ProbabilisticInference
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 88: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/88.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise on the data
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
ProbabilisticInference
![Page 89: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/89.jpg)
An examplefunction privBerInput (l: B list) (p1: R) (p2: R): M[(0,1)]{ let function vExp (l: B list) : M[B list]{ match l with |nil -> mreturn nil |x::xs -> coercion (exp eps((0,0)->1,(0,1)->0,(1,1)->1, (1,0)->0) x) :: (vExp l) } in mlet nl = (vExp l) in let prior = mreturn(beta(p1,p2)) in let function Ber (l: B list) (p:M[(0,1)]): M[(0,1)]{ match l with |nil -> ran(p) |x::xs -> observe y => y = x in (Ber xs p) } in mreturn(infer (Ber nl prior)) }
![Page 90: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/90.jpg)
An examplefunction privBerInput (l: B list) (p1: R) (p2: R): M[(0,1)]{ let function vExp (l: B list) : M[B list]{ match l with |nil -> mreturn nil |x::xs -> coercion (exp eps((0,0)->1,(0,1)->0,(1,1)->1, (1,0)->0) x) :: (vExp l) } in mlet nl = (vExp l) in let prior = mreturn(beta(p1,p2)) in let function Ber (l: B list) (p:M[(0,1)]): M[(0,1)]{ match l with |nil -> ran(p) |x::xs -> observe y => y = x in (Ber xs p) } in mreturn(infer (Ber nl prior)) }
Noise
![Page 91: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/91.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
Adding noise on the output
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
ProbabilisticInference
![Page 92: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/92.jpg)
DP for Probabilistic Programs
PosteriorDistribution
![Page 93: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/93.jpg)
DP for Probabilistic Programs
PosteriorDistribution
Releasing theParameters
![Page 94: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/94.jpg)
DP for Probabilistic Programs
PosteriorDistribution
Releasing theParameters
Sampling fromthe Distribution
![Page 95: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/95.jpg)
DP for Probabilistic Programs
PosteriorDistribution
Releasing theParameters
Sampling fromthe Distribution
![Page 96: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/96.jpg)
An exampleA distance over distributions
function privBerInput (l: B list) (p1: R) (p2: R): M[(0,1)]{ let function hellingerDistance (a0:R) (b0:R) (a1:R) (b1:R) : R { let gamma (r:R) = (r-1)! in let betaf (a:R) (b:R) = gamma(a)*gamma(b))/gamma(a+b) in let num=betaf ((a0+a1)/2.0) ((b0+b1)/2.0) in let denum=Math.Sqrt((betaf a0 b0)*(betaf a1 b1)) in Math.Sqrt(1.0-(num/denum)) } in let function score (input:M[(0,1)]) (output:M[(0,1)]) : R { let beta(a0,b0) = input in let beta(a1,b1) = output in (-1.0) * (hellingerDistance a0 b0 a1 b1) } in let prior = mreturn(beta(p1,p2)) in let function Ber (l: B list) (p:M[(0,1)]): M[(0,1)]{ match l with |nil -> ran(p) |x::xs -> observe y => y = x in (Ber xs p) } in exp eps score (infer (Ber l prior)) }
![Page 97: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/97.jpg)
Noise
An exampleA distance over distributions
function privBerInput (l: B list) (p1: R) (p2: R): M[(0,1)]{ let function hellingerDistance (a0:R) (b0:R) (a1:R) (b1:R) : R { let gamma (r:R) = (r-1)! in let betaf (a:R) (b:R) = gamma(a)*gamma(b))/gamma(a+b) in let num=betaf ((a0+a1)/2.0) ((b0+b1)/2.0) in let denum=Math.Sqrt((betaf a0 b0)*(betaf a1 b1)) in Math.Sqrt(1.0-(num/denum)) } in let function score (input:M[(0,1)]) (output:M[(0,1)]) : R { let beta(a0,b0) = input in let beta(a1,b1) = output in (-1.0) * (hellingerDistance a0 b0 a1 b1) } in let prior = mreturn(beta(p1,p2)) in let function Ber (l: B list) (p:M[(0,1)]): M[(0,1)]{ match l with |nil -> ran(p) |x::xs -> observe y => y = x in (Ber xs p) } in exp eps score (infer (Ber l prior)) }
![Page 98: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/98.jpg)
Noise
An exampleA distance over distributions
function privBerInput (l: B list) (p1: R) (p2: R): M[(0,1)]{ let function hellingerDistance (a0:R) (b0:R) (a1:R) (b1:R) : R { let gamma (r:R) = (r-1)! in let betaf (a:R) (b:R) = gamma(a)*gamma(b))/gamma(a+b) in let num=betaf ((a0+a1)/2.0) ((b0+b1)/2.0) in let denum=Math.Sqrt((betaf a0 b0)*(betaf a1 b1)) in Math.Sqrt(1.0-(num/denum)) } in let function score (input:M[(0,1)]) (output:M[(0,1)]) : R { let beta(a0,b0) = input in let beta(a1,b1) = output in (-1.0) * (hellingerDistance a0 b0 a1 b1) } in let prior = mreturn(beta(p1,p2)) in let function Ber (l: B list) (p:M[(0,1)]): M[(0,1)]{ match l with |nil -> ran(p) |x::xs -> observe y => y = x in (Ber xs p) } in exp eps score (infer (Ber l prior)) }
![Page 99: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/99.jpg)
More general Lifting of P
P(x1,x2) P*(x1,x2)
Relation over distributionsRelation
![Page 100: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/100.jpg)
Accuracy
Different ways of adding noise can have differentaccuracy.- we have theoretical accuracy (how to integrate it
in a framework for reasoning about DP?)- we have experimental accuracy (we need a
framework for for test our programs)
![Page 101: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/101.jpg)
ProgramProbabilistic
PriorDistribution
PosteriorDistribution
A Nice Playground!
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
ProbabilisticInference
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
Differential Privacy: the idea
A. Haeberlen
Promising approach: Differential privacy
3USENIX Security (August 12, 2011)
Private data
N(flue, >1955)?
826±10
N(brain tumor, 05-22-1955)?
3 ±700
Noise
?!?
Differential Privacy:Ensuring that the presence/absence of an individual has anegligible statistical effect on the query’s result.
Trade-off between utility and privacy.
![Page 102: Bayesian Inferencetarmo/tsem15/gaboardi0312-slides.pdf · Marco Gaboardi University of Dundee Verifying Differentially Private Bayesian Inference Joint work with G. Barthe, G.P. Farina,](https://reader033.vdocuments.us/reader033/viewer/2022042003/5e6e55c0a421c743bc79b271/html5/thumbnails/102.jpg)
Tänan teid väga!