Contextual Integrity & its Logical Formalization

18739A: Foundations of Security and Privacy

Anupam DattaFall 2009

Privacy in Organizations

2

Huge amount of personal information available to various organizations Hospitals, financial institutions, websites, universities, census

bureau, social networks… Information shared within and across organizational

boundaries to enable useful tasks to be completed Personal health information has to be shared to enable

diagnosis, treatment, billing, …

Fundamental Question: Do organizational processes respect privacy expectations while

achieving purpose?

Approach

What is Privacy?

Express and Enforce Privacy Policies

Contextual Integrity (CI):- Transfer of personal information governed by contextual norms

Logic of Privacy and Utility:- Formalization of CI; automated methods for checking compliance with privacy regulations (HIPAA, GLBA, COPPA,…)

Contextual Integrity [Nissenbaum 2004] Philosophical framework for privacy Central concept: Context

Examples: Healthcare, banking, education What is a context?

Set of interacting agents in roles Roles in healthcare: doctor, patient, …

Informational norms Doctors should share patient health information as per the

HIPAA rules Norms have a specific structure (descriptive theory) Systematic method at arriving at contextual norms

(prescriptive theory) Purpose

Improve health Some interactions should happen - patients should share

personal health information with doctors

Outline

1. Motivating Example2. Framework

Model Logic of Privacy and Utility

3. Workflows and Responsibility4. Algorithmic Results

Workflow Design assuming agents responsible

Auditing logs when agents irresponsible

5. Summary

MyHealth@Vanderbilt++ Workflow

Health Answer

Health Answer

Health Question

Appointment R

equest

Health Question

Now that I have cancer,Should I eat more vegetables?

HealthQuestion

Yes! except broccoli

HealthAnswer

Humans + Electronic system

Secretary

Nurse

DoctorPatien

t

Outline

1. Motivating Example2. Framework

Model Logic of Privacy and Utility

3. Workflows and Responsibility4. Algorithmic Results

Workflow Design assuming agents responsible

Auditing logs when agents irresponsible

5. Conclusions

Informational Norms

“In a context, the flow of information of a certain type about a subject (acting in a particular capacity/role) from one actor (could be the subject) to another actor (in a particular capacity/role) is governed by a particular transmission principle.”

Contextual Integrity [Nis2004]

Model

Alice

Communication via send actions: Sender: Bob in role Patient Recipient: Alice in role Nurse Subject of message: Bob Tag: Health Question Message: Now that ….

Data model & knowledge evolution: Agents acquire knowledge by:

receiving messages deriving additional attributes based on data model

Health Question Protected Health Information

BobNow that I have cancer,Should I eat more vegetables?

HealthQuestion

contents(msg) vs. tags (msg)

Model

State determined by knowledge of each agent

Transitions change state Set of concurrent

send actions Send(p,q,m)

possible only if agent p knows m

K0

K13

K11

......

K12

A11

A12

A13

Concurrent Game Structure

G = <k, Q, , , d, >

[Barth,Datta,Mitchell,Sundaram 2007]

Logic of Privacy and Utility

Syntax ::= send(p1,p2,m) p1 sends p2 message m

| contains(m, q, t) m contains attrib t about q

| tagged(m, q, t) m tagged attrib t about q

| inrole(p, r) p is active in role r | t t’ Attrib t is part of attrib t’ | | | x. Classical operators | U | S | O Temporal

operators | <<p>> Strategy quantifier

SemanticsFormulas interpreted over concurrent game structure

Informational Norms

“In a context, the flow of information of a certain type about a subject (acting in a particular capacity/role) from one actor (could be the subject) to another actor (in a particular capacity/role) is governed by a particular transmission principle.”

Contextual Integrity [Nis2004]

13

Privacy Regulation Example (GLB Act)

Financial institutions must notify consumers if they share their non-public personal information with non-affiliated companies, but the notification may occur either before or after the information sharing occurs

Exactly as CI says!

Sender role Subject role

Attribute

Recipient role

Transmission principle

Structure of Privacy Rules

Allow message transmission if:

•at least one positive norm is satisfied; and

•all negative norms are satisfied

[Barth,Datta,Mitchell,Nissenbaum 2006]

HIPAA – Healthcare Privacy

•HIPAA consists primarily of positive norms: share phi if some rule explicitly allows it (2), (3), (5), (6)

•Exception: negative norm about psychotherapy notes (4)

COPPA – Children Online Privacy

•COPPA consists primarily of negative norms

•children can share their protected info only if parents consent (7) (condition)

•(8) (obligation – future requirements)

Specifying Privacy

MyHealth@Vanderbilt

In all states, only nurses and doctors receive health questions

G p1, p2, q, m send(p1, p2, m) contains(m, q, health-

question) inrole(p2, nurse) inrole(p2, doctor)

Specifying Utility MyHealth@Vanderbilt

Patients have a strategy to get their health questions answered

p inrole(p, patient) <<p>> F q, m.

send(q, p, m) contains(m, p, health-answer)

Outline1. Motivating Example2. Framework

Model Logic of Privacy and Utility

3. Workflows and Responsibility4. Algorithmic Results

Workflow Design assuming agents responsible

Auditing logs when agents irresponsible

5. Conclusions

6. Differentially Private Workflows (WIP)

Nurse

Secretary

MyHealth@Vanderbilt Improved

Patient

Doctor

Health Answer

Health Answer

Health Question

Appointment R

equest

Health Question

Now that I have cancer,Should I eat more vegetables?

HealthQuestion

Yes! except broccoli

HealthAnswer

Assign responsibilities to roles & workflow engine

Doctor should answer health questions

Graph-based Workflow

Graph (R, R x R), where R is the set of roles

Edge-labeling function permit: R x R 2T, where T is the set of attributes

Responsibility of workflow engineAllow msg from role r1 to role r2 iff tags(msg) permit(r1, r2)

Responsibility of human agents in rolesTagging responsibilities

ensure messages are correctly taggedProgress responsibilities

ensure messages proceed through workflow

MyHealth Responsibilities Tagging

Nurses should tag health questionsG p, q, s, m. inrole(p, nurse) send(p, q, m)

contains(m, s, health-question) tagged(m, s, health-question)

Progress Doctors should answer health questions

G p, q, s, m. inrole(p, doctor) send(q, p, m) contains(m, s, health-question)

F m’. send(p, s, m’) contains(m’, s, health-answer)

Abstract Workflow Responsibility of workflow engine

LTL formula Feasible (enforceable) if is a safety formula

without the contains() predicate

Responsibility of each role r LTL formula r

Feasible if agents have a strategy to discharge their responsibilities

p. inrole(p, r) <<p>> r

Graph-based workflows are a special case

Outline1. Motivating Example2. Framework

Model Logic of Privacy and Utility

3. Workflows and Responsibility4. Algorithmic Results

Workflow Design assuming agents responsible

Abstract workflows Auditing logs when agents irresponsible

Only graph-based workflows

5. Conclusions

Workflow Design Results Theorems:

Assuming all agents act responsibly, checking whether workflow achieves

Privacy is in PSPACE (in the size of the formula describing the workflow)

Utility is decidable

Algorithms implemented in model-checkers, e.g. SPIN, MOCHA

Potential for small model theorems

Outline1. Motivating Example2. Framework

Model Logic of Privacy and Utility

3. Workflows and Responsibility4. Algorithmic Results

Workflow Design assuming agents responsible

Abstract workflows Auditing logs when agents irresponsible

Only graph-based workflows

5. Summary

Auditing Results: Summary

Definitions Accountability = Causality + Irresponsibility

Design of audit log Captures causality relation

Algorithm Finding agents accountable for policy violation

in graph-based workflows using audit log Algorithm uses oracle:

O(msg) = contents(msg)

Policy compliance/violation

Strong compliance [BDMN2006, BDMS 2007] Action does not violate current policy requirements Future policy requirements after action can be met Formally,

Policy

History

Contemplated ActionJudgment

Future Reqs

Local Compliance

Locally compliant policy Agents can determine strong compliance based on

their local view of history

Causality Lamport Causality [1978]

Accountability & Audit Log Accountability

Causality + Irresponsibility Audit log design

Records all Send(p,q,m) and Receive(p,q,m) events executed

Maintains causality structure O(1) operation per event logged

Auditing Algorithm

Central Lemma

Main Theorem

MyHealth Example

1. Policy violation: Secretary Candy receives health-question

mistagged as appointment-request

2. Construct causality graph G and search backwards using BFS

Candy received message m from Patient Jorge.

O(m) = health-question, but tags(m) = appointment-request.

Patient responsible for health-question tag. Jorge identified as accountable

Summary

1. Framework Concurrent game model Logic of Privacy and Utility

Temporal logic (LTL, ATL*)

2. Organizational Process as Workflow Role-based responsibility for human and

mechanical agents

3. Algorithmic Results Workflow design assuming agents

responsible Privacy, utility decidable (model-checking)

Auditing logs when agents irresponsible From policy violation to accountable agents

Using oracle

Automated

Thanks

Questions?

Deciding Privacy PLTL model-checking problem is PSPACE

decidable

G |= tags-correct U agents-responsible privacy-policy

G: concurrent game structure

Result applies to finite models (#agents, msgs,…)

Deciding Utility ATL* model-checking of concurrent game

structures is Decidable with perfect information Undecidable with imperfect information

Theorem:There is a sound decision procedure for deciding

whether workflow achieves utility

Intuition: Translate imperfect information into perfect

information by considering possible actions from one player’s point of view

Local communication game Quotient structure under invisible actions, Gp

States:Smallest equivalence relation K1 ~p K2 if K1 K2 and a is invisible to p

Actions:[K] [K’] if there exists K1 in [K] and K2 in [K’] s.t. K1 K2

Lemma: For all LTL formulas visible to p, Gp |= <<p>> implies G |= <<p>>

The truth value of formulas visible to p depends only on actions that p can observe, e.g. send (*, p, *), send(p, *, *)Locality again!

Related Languages

Model Sender Recipient Subject Attributes Past FutureCombinatio

n

RBAC Role Identity

XACML Flexible Flexible Flexible o o

EPAL Fixed Role Fixed o

P3P Fixed Role Fixed o o

LPU Role Role Role

Legend: unsupportedo partially supported fully supported

LPU fully supports attributes, combination, temporal conditions, and utility