ZQL a cryptographic compiler for processing private data

George DanezisCédric Fournet Markulf KohlweissZhengqin Luo & Alfredo Rial

• Privacy at odds with big dataproduced, processed, and stored

• Private data?– Personal, medical, financial, legal…

• A controversial trust issue – Show-stopper when

deploying new technology– High potential for negative press– Strong EU regulations

• Wanted: generic tools forprivacy-friendly data processing(as in dolphin-friendly tuna)

Data Privacy

Example 1:Smart Metering

Privacy-Preserving Smart Metering

• Only the monthly fee need to be sent back to the utility(not the detailed meter readings)

certified readings(private data)

Utility Provider

User, payingher monthly bill

price to pay+ crypto evidence

Smart Meter

variable policy & rates

Example 2: Pay-how-you-drive insurance

Example 2: Pay-how-you-drive insurance

Example 2: Pay-how-you-drive insurance

(…)

Pay-how-you-drive insurance

• Only the premium needs to be communicated to the insurance company

User, payingher insurance fee

fee to pay+ crypto evidence

certified pricing policy

certified location, speed and distance(private data)

Insurance company

Many similar problems

• (Partly) private user data• Public pricing policy• Conflicting goals:

Privacy for the user• Concealing meter readings• Concealing locations, speed and distance

Integrityfor the verifier

Only the correct pricecan be proved

So far, ad hoc cryptographic solutions

• Bespoke privacy-preserving protocolsusing a mix of cryptographic mechanisms– Linear policies using homomorphic commitments– Cumulative policies using zero-knowledge proofs on CL-signatures– Penalty-based policies (Pay-how-you-drive) ?

• Also many protocols for anonymous credentials, e-cash, e-voting

• These protocols are available today,but they hard to design, implement, and deploy– We cannot involve cryptographers and security experts

each time we change the query or revise the service policy!

Goal: private data processing

1. a high-level language for querying data2. an optimizing, verifying query compiler– selects cryptographic constructions– generates code for different platforms– verifies its security before deployment

certifiedprivate data

certified public data

Service, or Verifierin need of

valid results

Client, or Proverin control of her data

query results+ cryptoevidence

ReliableData Providers

SELECT * FROM … WHERE …

agreement ona data query

Integrity and Privacy (Ideal)

Client, or Proverin control of her data

Trusted Third Party

ReliableData Providers

SELECT * FROM … WHERE …

“Ok.”

Service, or Verifierin need of

valid results

ZQL: a language for querying private data

• SQL [Structured Query Language, 1970—]:a fine declarative domain-specific languagefor querying relational DBs

• ZQL [Zero-Knowledge Query Language]: a subset of SQL extended for cryptographic processing– privacy annotations– random sampling– hash, sign,… – big numbers for keys, group elements, exponents, …

• The SQL theory carries over to ZQL, despite unusual data: – Useful algebraic properties – Efficient evaluation plans and representations (e.g. indexing)

fast, portablelow-level code in C

queries expressed

in SQL+crypto

query expressed in SQL

ZQL compiler

We emit code for each participant

referencehigh-level code in F#

data privacyspecification

F# generator

We generate a queryfor each participant,with matching I/Os

Q(T1…Tn)

Tsign1 QproveQverify

T1 : …

Qv.cQp.c

T1.cQv.fsQp.fs

T1.fs

C generator

Compiler Architecture

crypto.fsZQL.fs

runtime libraries

Sample SQL query for Smart MeteringConsider billing

with a variable-ratelinear policy

Meter

time reading

12/1 00:00 241

12/1 00:15 322

12/1 00:30 325

12/1 00:45 116

12/1 01:00 538

…

Policy

time rate

12/1 00:00 12

12/1 00:15 12

12/1 00:30 15

12/1 00:45 15

12/1 01:00 15

…

Bill := SELECT time, cost = reading * rate FROM Meter |><| Policy WHERE time in 1/1..31/1/2011

fee := SUM cost FROM Bill

The source query is:

(2) the client now computestwo sums instead of one

Commit

time commit

Policy

time rate

public signed data secret data+ Open

reading opening

(1) the meter generatesand signs commitments

Inside ZQL:

Compiling the metering query

public rates

Open := SELECT time, reading, opening = RNG() FROM MeterCommit:= SELECT time, commit = g^reading * h^opening FROM Open

g^payment * h^proof

= g^(SUM (reading * rate)) * h^(SUM (opening * rate))= PROD (g^reading * h^opening)^rate= PROD commit^rate

payment := SUM reading * rate FROM Open |><| Policyproof := SUM opening * rate FROM Open |><| Policy

g^payment * h^proof =?= PROD commit ^ rate FROM Commit |><| Policy

(3) the utility checks them against the signed commitments

Inside ZQL:

Compiling pay-how-you-drive (outline)This query joins tables on secret columns: we must prove that the join includes rows from the policy tableswithout revealing which rows we actually use

GPS private readings

time road speed mileageSpeedLimit

road max

Penalty

over points

Policy

time points rate

let Over = SELECT over = speed - max FROM GPS |><| SpeedLimit

let total = SUM points FROM Over |><| Penalty

let payment = SUM mileage * rate FROM GPS |><| Policy WHERE points = total

GPS private readings

time road speed mileage opening commit

This query joins tables on secret columns: we must prove that the join includes rows from the policy tableswithout revealing which rows we actually useFor each row in a join, the prover generates a zero-knowledge proof of knowledgeof a matching signed row in the policy table

SpeedLimit

road max CLsig

Penalty

over points CLsig

Policy

time points rate CLsig

Inside ZQL:

Compiling pay-how-you-drive (outline)

let Over = SELECT over = speed - max FROM GPS |><| SpeedLimit

let total = SUM points FROM Over |><| Penalty

let payment = SUM mileage * rate FROM GPS |><| Policy WHERE points = total

Expressiveness & Performance

• When operating on secrets,we support a fragment of SQL1. Linear expressions2. Polynomials expressions3. Table lookups4. Inequalities

• Current limitation: The shape of intermediatetables must be public

• Fresh cryptography:vector commitments for implementing privateintermediate tables

• Linear overheadin the number of secretsLinear:

128 bytes/rowprove 300,000 row/Scheck 90,000 row/S

Non-Linear:6586 bytes/row

prove 87 row/Scheck 41 row/S

• Performance dominatedby bignum multiplications:– We rely on recent extensions

of core crypto algorithms(thanks to XCG)

Security?

• Our compiler automaticallygenerate complex protocolimplementations– Hard to test or review– How can the user tell whether

her privacy is preserved?

• Our compiler then calls independent, automatedtools to verify that theseimplementations are secure– Privacy: the service learns nothing more than the query result– Integrity: the user can build evidence only for the correct result

F# generator

Qv.fsQp.fs

T1.fs

ZQL compiler

Q(T1…Tn)T1: …

Security Verification (2003—)

• We develop a cryptographic verification kitfor new protocol implementations [with A.D. Gordon, K. Bhargavan]– Tools: ProVerif, FS2PV, FS2CV, F7, F*…– Mostly for F#, with experiments for C and C#

• We automatically verify large implementationsagainst precise cryptographic assumptions– Probabilistic security: “no secret information flows to the adversary”– Computational security: “except with a negligible probability, no adversary can…”

• Verification case studies:– TLS 1.2 Internet Standard [with MSR-INRIA]; web services security– DKM for cloud data [with T. Acar, D.Shumow] deployed on MS datacenters– TPM2 [with P. England, D. Wooten, F. Dupressoir, and RISE] shipping with Win8

typed specification in F7 reference implementation in F#

F# generator

Qv.fsQp.fs

T1.fs

F7 generator

Qv.fs7Qp.fs7

T1.fs7

F7 typing

We get either a compile-time error (bug)or strong integrity & privacy theorems

ZQL compiler

Q(T1…Tn)T1: …

Crypto.fs7

ZQL.fs7

Towards a certifying ZQL Compiler

We generate proof goalsand type annotations to keep track of query evaluations

We use F7to automatically prove that implementations conform with their specifications

ZQL: Privacy-Friendly Data Processing

1. With ZQL, clients process their own private data & services still get correct results

2. Programmers specify their SQL queries & privacy goals;we compile them into zero-knowledge protocols

3. The security of fresh crypto implementationscan be automatically verified (at compile-time)under standard security assumptions

©2011 Microsoft Corporation. All rights reserved.