Los Alamos National Laboratory

Sample-Optimal Learning of Graphical Models(and opening the D-Wave annealer’s quantum box)

Andrey Lokhovwith M. Chertkov, C. Coffrin, Y. Kharkov, S. Misra, M. Vuffray

Theoretical Division, Los Alamos National Laboratory

Learning General Markov Random Fields Performance and Rigorous Guarantees

Quantum Annealing and Inverse Ising Problem Understanding the Behavior of D-Wave Annealer

𝜎𝑖 𝜎𝑗𝐽𝑖𝑗𝜎𝑖𝜎𝑗

Input Output

𝐽𝑖𝑗Couplings

Fields ℎ𝑖 𝜎

Annealing

Spin configurations

𝐽1

𝐽2

𝐽𝑠Indistinguishable

Distributions

𝐽1

𝐽2

𝜉

𝜉

𝜉

)( kkf

Arbitrary alphabet

Arbitrary interaction orders

qAA iii ,

Lk

Arbitrary basis functions )( kkf

iKk

ki

k

Kk

kkk fZ

***max,min))(exp

1)(

Example: binary variables and multi-body interactions

Learning problem: given M i.i.d. samples , with probability learn the graph of conditional dependencies and estimate couplings

Regularized Interaction Screening Estimator (RISE): convex optimization

Optimal Structure and Parameter Learning of Ising Models

A. Lokhov, M. Vuffray, S. Misra, M. Chertkov

Interaction Screening: Efficient and Sample-Optimal Learning of Ising Models

M. Vuffray, S. Misra, A. Lokhov, M. Chertkov

(2018)

(2016)

Theorem : If , then with prob.

Our Method: Interaction Screening

kj

ijk

iijkj

ij

iij

i

ii JJh ***

exp)(

1

,

expminargˆ,ˆ ikj

jk

iijkj

j

iijiihJ

ii JJJhhJii

NqCM

L

ln)6exp(

2

2

2

ˆ2

* ii 1

i

1 )()1(,,

M

*

Learning Discrete Graphical Models with Generalized Interaction Screening

S. Misra, A. Lokhov, M. Vuffray (submitted)

Exact, tractable & parallelizable algorithm

Requires near IT optimal number of samples

Works for any models: low temperature, spin glass, …

Comparison with IT bounds for Ising models:

Generalizations: dense models, reconstruction in the dynamic setting, …

+ assumptions-free proof for logistic regression loss (pseudo-likelihood)

Classical annealing Quantum annealing

Tem

per

atu

re

Ob

ject

ive

zx sHHssH )1()(

D-Wave computer is an implementation of quantum annealing for Ising models

Motivation: assessing performance and quantify uncertainties

Solving Inverse Ising problem

Question 2: What can we learn about the machine from this distribution?

• Type of interactions?• Structure of interactions? • Input-output response?

Question 1: Characterize the probability distribution of output data

Mixture of models withfluctuating magnetic fields

Single model with aspurious coupling

Statistical reconstruction of instantaneous field fluctuations spurious links with quadraticresponse

Spurious interactions arequadratic response to physicalinteractions forming “triangles”

Explanation for spurious links:

Structure of distribution:

Input-output response:

Opening the Quantum Box: Understanding the Behavior of Analogue Annealers

with Statistical Learning A. Lokhov, Y. Kharkov, C. Coffrin, M. Vuffray (submitted)

56 Three-Body Terms