anupam datta anupam dattacmu joint work with adam barth, john mitchell (stanford), helen nissenbaum...
Post on 21-Dec-2015
222 views
TRANSCRIPT
Anupam DattaAnupam Datta
CMUCMU
Joint work with Adam Barth, John Mitchell Joint work with Adam Barth, John Mitchell (Stanford), Helen Nissenbaum (NYU) and Sharada (Stanford), Helen Nissenbaum (NYU) and Sharada Sundaram (TCS)Sundaram (TCS)
Privacy and Contextual Integrity:Framework and Applications
2
Problem Statement
Is an organization’s business process compliant with privacy regulations and internal policies?
Examples of organizations– Hospitals, financial institutions, other enterprises handling sensitive
information Examples of privacy regulations
– HIPAA, GLBA, COPPA, SB1386
Goal: Develop methods and tools to answer this question
3
Privacy Project Space
What is Privacy?[Philosophy, Law, Public Policy]
Formal Model, Policy Language,Compliance-check Algorithms
[Programming Languages, Logic]
Implementation-level Compliance[Software Engg, Formal Methods]
Data Privacy[Databases, Cryptography]
4
Project Overview
What is privacy?– Conceptual framework
Policy language – Privacy laws including HIPAA, COPPA, GLBA expressible
Compliance-check algorithms– Does system satisfy privacy and utility goals?
Case studies– Patient portal deployed at Vanderbilt Hospital– UPMC (ongoing discussions)– TCS
5
Contextual Integrity [N2004]
Philosophical framework for privacy Central concept: Context
– Examples: Healthcare, banking, education What is a context?
– Set of interacting agents in roles Roles in healthcare: doctor, patient, …
– Norms of transmission Doctors should share patient health information as per the HIPAA rules
– Purpose Improve health
6
Nurse
Secretary
MyHealth@Vanderbilt Workflow
Patient
Doctor
Health Answer
Health AnswerHealth Question
Appointment R
equest
Healt
h Q
uest
ion
Health Questio
n
Now that I have cancer,Should I eat more vegetables?
Yes! except broccoli
Privacy: HIPAA compliance+
Humans + Electronic system
Utility: Schedule appointments, obtain health answers
7
Nurse
Secretary
MyHealth@Vanderbilt Improved
Patient
Doctor
Health Answer
Health Answer
Health Question
Appointment R
equest
Health Question
Now that I have cancer,Should I eat more vegetables?
HealthQuestion
Yes! except broccoli
HealthAnswer
•Message tags used for policy enforcement
•Minimal disclosure
Responsibility: Doctor should answer health questions
8
Privacy vs. Utility
Privacy– Certain information should not
be communicated Utility
– Certain information should be communicated
Tension between privacy and utility
Perm
issiveness
Workflows
ViolatePrivacy
ViolateUtility
FeasibleWorkflows
“Minimum necessary”
9
Design-time Analysis: Big Picture
ContextualIntegrity
Business Objectives Privacy Policy
Business ProcessDesign
PrivacyChecker
(LTL)
UtilityChecker(ATL*)
UtilityEvaluation
PrivacyEvaluation
NormsPurpose
Assuming agents responsible
10
Auditing: Big Picture
Business ProcessExecution
AuditLogs
Run-time Monitor
Privacy PoliciesUtility Goals
AuditAlgos
Policy Violation+
Accountable Agent
Agents may not be responsible
11
In more detail…
Model and logicPrivacy policy examples
– GLBA – financial institutions– MyHealth portal
Compliance checking– Design time analysis (fully automated)– Auditing (using oracle)
Language can express HIPAA, GLBA, COPPA [BDMN2006]
12
Model
Alice
– Communication via send actions: Sender: Bob in role Patient Recipient: Alice in role Nurse Subject of message: Bob Tag: Health Question Message: Now that ….
– Data model & knowledge evolution: Agents acquire knowledge by:
– receiving messages – deriving additional attributes based on data model
Health Question Protected Health Information
BobNow that I have cancer,
Should I eat more vegetables?
HealthQuestion
contents(msg) vs. tags (msg)
Inspired by Contextual Integrity
13
Model
State determined by knowledge of each agent
Transitions change state– Set of concurrent send actions– Send(p,q,m) possible only if
agent p knows m
K0
K13
K11
......
K12
A11
A12
A13
Concurrent Game Structure
G = <k, Q, , , d, >
[BDMN06, BDMS07]
14
Logic of Privacy and Utility
Syntax ::= send(p1,p2,m) p1 sends p2 message m
| contains(m, q, t) m contains attrib t about q | tagged(m, q, t) m tagged attrib t about q | inrole(p, r) p is active in role r | t t’ Attrib t is part of attrib t’ | | | x. Classical operators | U | S | O Temporal operators
| <<p>> Strategy quantifier Semantics
Formulas interpreted over concurrent game structure
15
Specifying Privacy
MyHealth@Vanderbilt
In all states, only nurses and doctors receive health questions
G p1, p2, q, m
send(p1, p2, m) contains(m, q, health-question)
inrole(p2, nurse) inrole(p2, doctor)
16
Specifying Utility
MyHealth@Vanderbilt
Patients have a strategy to get their health questions answered
p inrole(p, patient)
<<p>> F q, m. send(q, p, m) contains(m, p, health-answer)
17
MyHealth Responsibilities
Tagging Nurses should tag health questions
G p, q, s, m. inrole(p, nurse) send(p, q, m) contains(m, s, health-question)
tagged(m, s, health-question) Progress
– Doctors should answer health questionsG p, q, s, m. inrole(p, doctor) send(q, p, m)
contains(m, s, health-question) F m’. send(p, s, m’) contains(m’, s, health-answer)
18
Sender role Subject roleAttribute
Temporal condition
Gramm-Leach-Bliley Example
Recipient role
Financial institutions must notify consumers if they share their non-public personal information with non-affiliated companies, but the notification may occur either before or after the information sharing occurs
19
Workflow Design Results
Theorems:Assuming all agents act responsibly, checking whether workflow
achieves – Privacy is in PSPACE (in the size of the formula describing the
workflow) Use LTL model-checking algorithm
– Utility is decidable for a restricted class of formulas ATL* model-checking is undecidable for concurrent game structures with
imperfect information, but decidable with perfect information
Idea: – Check that all executions satisfy privacy and utility properties
Definition and construction of minimal disclosure workflow
Algorithms implemented in model-checkers, e.g. SPIN, MOCHA
20
Auditing Results
Who to blame? Accountability– Irresponsibility + causality
Design of audit log– Use Lamport causality structure, standard concept from distributed
computing Algorithms
– Finding agents accountable for policy violation in graph-based workflows using audit log
– Finding agents who act irresponsibly using audit log Algorithms use oracle:
– O(msg) = contents(msg)– Minimize number of oracle calls
21
Conclusions
Framework inspired by contextual integrity Business Process as Workflow
Role-based responsibility for human and mechanical agents Compliance checking
Workflow design assuming agents responsible Privacy, utility decidable (model-checking) Minimal disclosure workflow constructible
Auditing logs when agents irresponsible From policy violation to accountable agents Finding irresponsible agents
Case studies– MyHealth patient portal deployed at Vanderbilt University hospital– Ongoing interactions with UPMC
Using oracle
Automated
22
Future Work
Framework– Do we have the right concepts? – Adding time , finer-grained data model– Priorities of rules, inconsistency, paraconsistency
Compliance vs. risk management Privacy principles
– Minimum necessary one example; what else? Improve algorithmic results
– Utility decidability; small model theorem– Auditing algorithms
Privacy analysis of code– Current results apply to system specification
More case studies– Focusing on healthcare
Detailed specification of privacy laws– Immediate focus on HIPAA, GLBA, COPPA
Legal, economic incentives for responsible behavior
23
Publications/Credits
– A. Barth, A. Datta, J. C. Mitchell, S. Sundaram
Privacy and Utility in Business Processes, to appear in Proceedings of 20th IEEE Computer Security Foundations Symposium, July 2007.
– A. Barth, A. Datta, J. C. Mitchell, H. Nissenbaum
Privacy and Contextual Integrity: Framework and Applications, in Proceedings of 27th IEEE Symposium on Security and Privacy , pp. 184-198, May 2006.
Work covered in The Economist, IEEE Security &
Privacy editorial
26
Related Languages
Model Sender Recipient Subject Attributes Past Future Combination
RBAC Role Identity
XACML Flexible Flexible Flexible o o
EPAL Fixed Role Fixed o
P3P Fixed Role Fixed o o
LPU Role Role Role
Legend: unsupportedo partially supported fully supported
LPU fully supports attributes, combination, temporal conditions
Utility not considered
27
Deciding Utility
ATL* model-checking of concurrent game structures is – Decidable with perfect information– Undecidable with imperfect information
Theorem:There is a sound decision procedure for deciding whether workflow
achieves utility Intuition:
– Translate imperfect information into perfect information by considering all possible actions from one player’s point of view
28
Local communication game
Quotient structure under invisible actions, Gp
– States:Smallest equivalence relation
K1 ~p K2 if K1 K2 and a is invisible to p – Actions:
[K] [K’] if there exists K1 in [K] and K2 in [K’] s.t. K1 K2
Lemma: For all LTL formulas visible to p, Gp |= <<p>> implies G |= <<p>>
29
Auditing Results
Definitions– Policy compliance, locally compliant– Causality, accountability
Design of audit log Algorithms
– Finding agents accountable for locally-compliant policy violation in graph-based workflows using audit log
– Finding agents who act irresponsibly using audit log Algorithms use oracle:
– O(msg) = contents(msg)– Minimize number of oracle calls
30
Policy compliance/violation
Strong compliance [BDMN2006]– Action does not violate current policy requirements– Future policy requirements after action can be met
Locally compliant policy– Agents can determine strong compliance based on their local view of history
Policy
History
Contemplated ActionJudgment
Future Reqs
32
Accountability & Audit Log
Accountability– Causality + Irresponsibility
Audit log design– Records all Send(p,q,m) and Receive(p,q,m) events executed– Maintains causality structure
O(1) operation per event logged
33
Auditing Algorithm
GoalFind agents accountable for a policy violation
Algorithm(Audit log A, Violation v)1. Construct G, the causality graph for v in A2. Run BFS on G.
At each Send(p, q, m) node, check if tags(m) = O(m). If not, and p missed a tag, output p as accountable
Theorem: – The algorithm outputs at least one accountable agent for every
violation of a locally compliant policy in an audit log of a graph-based workflow that achieves the policy in the responsible
model
34
Proof Idea
Causality graph G includes all accountable agents– Accountability = Causality + Irresponsibility
There is at least one irresponsible agent in G– Policy is satisfied if all agents responsible – Policy is locally compliant
In graph-based workflows, safety responsibilities violated only by mistagging
– O(msg) = tags(msg) check identifies all irresponsible actions
35
MyHealth Example
1. Policy violation: Secretary Candy receives health-question mistagged as
appointment-request
2. Construct causality graph G and search backwards using BFS Candy received message m from Patient Jorge. O(m) = health-question, but tags(m) = appointment-request. Patient responsible for health-question tag. Jorge identified as accountable