powerpoint file available: ~jamesh/powerpoint/ oslo_2010.ppt

PowerPoint File available:

http://bl831.als.lbl.gov/

~jamesh/powerpoint/

Oslo_2010.ppt

http://ucxray.berkeley.edu/~jamesh/elves

Download Elves from:

Advanced Light Source

Beamline 8.3.1 staff

Acknowledgments

George MeigsGeorge Meigs

Jane Tanamachi

ALS 8.3.1 Team

Acknowledgements

8.3.1 PRT: Jamie CateCenter for Structure of Membrane Proteins

Membrane Protein Expression Center IICenter for HIV Accessory and Regulatory Complexes

W. M. Keck FoundationPlexxikon, Inc.

M D Anderson CRCUniversity of California Berkeley

University of California San FranciscoNational Science Foundation

University of California Campus-Laboratory Collaboration GrantHenry Wheeler

The Advanced Light Source is supported by the Director, Office of Science, Office of Basic Energy Sciences, Materials Sciences Division, of the US Department of Energy under contract No. DE-AC02-05CH11231 at Lawrence Berkeley National Laboratory.

Ken Frankel Chris Neilson Michael Blum Joe Ferrara

Elves examine images andset-up data processing

Elves run…

mosflmscalasolve

mlpharedm

arp/warp

Elven Automation

Elven Automation

Elves examine images andset-up data processing

Elves run…

mosflmscalasolve

mlpharedm

arp/warp

Conversational User Interface

user input ->


user input -> process the data in /data/semet



recognition blah blah blah blah /data/semet




simplify /data/semet




simplify /data/semet

preempt /data/semet contains image files



report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 100 mm from the crystal using 1.54 Å x-rays.




confirm Everything look okay? [Yes] ->




confirm Everything look okay? [Yes] ->user input distance was more like 110


user input -> distance was more like 110



recognition distance blah blah blah 110



recognition distance blah blah blah 110

simplify DISTANCE 110




confirm Everything look okay? [Yes]user input -> wavelength is wrong


user input -> wavelength is wrong



recognition wavelength = -NOT




simplify WAVELENGTH




query What is the x-ray wavelength? [1.54Å] ->

simplify WAVELENGTH




query What is the x-ray wavelength? [1.54Å]user input -> 1

simplify WAVELENGTH



report Elves will process from /data/semet/frame001.img to /data/semet/frame100.img Data were collected from 0º to 100º in 1º steps with an ADSC Q4 detector 110 mm from the crystal using 1 Å x-rays.






confirm Everything look okay? [Yes]user input -> Yes

Major Phasing techniques

• Molecular Replacement

• Multiple Isomorphous Replacement

• Multiwavelength Anomalous Diffraction

• Single-wavelength Anomalous Diffraction

?

Molecular Replacementcorrect structure and intensities

http://www.ysbl.york.ac.uk/~cowtan/fourier/coeff.html

Molecular Replacementuse something similar as a starting model

Model Buildingcurrent model is missing something

Model Buildingphases from model

Model Buildingmissing bits show up in “difference map”

Model Buildingmissing bits show up better in FO + (FO - FC) map

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

1 param

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

1 param

3 params

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

data

1 param

3 params

10 params

stru

ctu

re f

acto

r (F

)

spot index (h)

Fitting data

inverse Fourier Transformno phase

dete

ctor

anomalous scattering

sam

ple

x-ray beam

dete

ctoranomalous scattering

sam

ple

x-ray beam

Independent tasks can be performed simultaneously

Multiprocessing Strategy

SOLVE/P212121/doneSOLVE/P21212/doneSOLVE/P21221/doneSOLVE/P22121/doneSOLVE/P2221/doneSOLVE/P2212/doneSOLVE/P2122/busySOLVE/P222/done

Multiprocessing StrategyMultiprocessing Strategy

epmr/P212121/model1/doneepmr/P212121/model2/doneepmr/P21212/model1/doneepmr/P21212/model2/doneepmr/P21221/model1/doneepmr/P21221/model2/doneepmr/P22121/model1/busyepmr/P22121/model2/doneepmr/P2221/model1/epmr/P2221/model2/epmr/P2212/model1/epmr/P2212/model2/epmr/P2122/model1/epmr/P2122/model2/


wARP/P212121/donewARP/P21212/donewARP/P21221/donewARP/P22121/donewARP/P2221/donewARP/P2212/donewARP/P2122/busywARP/P222/done


space group

FOM Rcryst

P3221

P3121

P321

Multiprocessing Advantage

space group

FOM Rcryst

P3221 0.836 0.323

P3121

P321


space group

FOM Rcryst

P3221 0.836 0.323

P3121 0.865 0.196

P321 0.726 0.347


table1.com

Elven Automation

How often does it really work?

Apr 6 – 24 at ALS 8.3.1

Elven Automation

27,686 images collected

Apr 6 – 24 at ALS 8.3.1

Elven Automation


148 datasets (15 MAD)

Apr 6 – 24 at ALS 8.3.1

Elven Automation



31 investigators

Apr 6 – 24 at ALS 8.3.1

Elven Automation



31 investigators

56 unique cells

Apr 6 – 24 at ALS 8.3.1

Elven Automation



31 investigators

56 unique cells

5 KDa – 23 MDa asymmetric unit

Apr 6 – 24 at ALS 8.3.1

Elven Automation



31 investigators

56 unique cells

5 KDa – 23 MDa asymmetric unit

0.94 – 32 Å resolution (3.2 Å)

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours

2 / 15 MAD structures

Number Description Percent

446028 Images (~7 TB) 33%

2346 Data sets 47%

449 MAD/SAD (1:2) 19%

104 Published 4.4%

8.3.1 in 2003

How many structures get solved?

Why do structures fail?

Overlaps


Overlaps

Signal to noise


Overlaps

Signal to noise

Radiation Damage


Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours


unavoidable overlaps


dete

ctor


phi

dete

ctor


mosaicity

phi

dete

ctor


mosaicity

phi

dete

ctor

c*


mosaicity

phi

dete

ctor

c*

Ewald sphere


mosaicity

phi

dete

ctor

c*

b

c

a

avoidable overlaps

mosaicity

phi

dete

ctor

c*

b

ca

avoidable overlaps

mosaicity

phi

dete

ctor

c*

bc

a

avoiding overlaps

avoiding overlaps

c

c

avoiding overlaps

1000 mm

avoiding overlaps

1000 mm

2 mrad

10 seconds

avoiding overlaps

1000 mm

2 mm

2 mrad

10 seconds

avoiding overlaps

1000 mm

1 mm

1 mrad

20 seconds

avoiding overlaps

1000 mm

300 um

0.3 mrad

60 seconds

Overlaps

Signal to noise

Radiation Damage


Apr 6 – 24 at ALS 8.3.1

Elven Automation

148 datasets

117 succeded

~3.5 (0.1-75) hours

31 failed

~61 (0-231) hours


Radiation Damage

why not just avoid it?

Holton & Frankel (2010) Acta D 66 393-408.

B ≈ 4 d2 + 120

20

4

0

60

8

0

100

12

0

aver

age

atom

ic B

fac

tor

1 1.5 2 2.5 3 3.5 4 4.5 5

resolution (Å)

Simulated diffraction imageSimulated diffraction imageMLFSOMMLFSOM

simulatedsimulated realreal

signal vs noise

signal vs noise

“If you don’t have good data,

then you have no data at all.”

-Sung-Hou Kim

signal vs noiseeasy

hard

impossible

signal vs noiseeasy

hard

impossible

threshold of “solvability”

signal vs noise

“If you don’t have good data,

then you must learn statistics.”

-James Holton

Adding noise

Adding noise

12 + 12 = 1.42

Adding noise

12 + 12 = 1.42

32 + 12 = 3.22

σtotal2 = σ1

2 + σ22

Adding noise

12 + 12 = 1.42

32 + 12 = 3.22

102 + 12 = 10.052

MAD phasing simulation

-0.2

0

0.2

0.4

0.6

0.8

1

0.01 0.1 1 10

Anomalous signal to noise ratio

Cor

rela

tion

coef

ficie

nt t

o co

rrec

t m

odel

mlphare results

SAD phasing simulation

-0.2

0

0.2

0.4

0.6

0.8

1

0.01 0.1 1 10


Cor

rela

tion

coef

ficie

nt t

o co

rrec

t m

odel

mlphare results

SAD phasing experiment

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.5 1.0 1.5 2.0 2.5 3.0


Cor

rela

tion

coef

ficie

nt t

o pu

blis

hed

mod

el

MR simulation

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.001 0.01 0.1 1 10 100 1000

Signal to noise ratio

Cor

rela

tion

coef

ficie

nt t

o co

rrec

t de

nsity

corrupted data

“photon counting”

Read-out noise

Shutter jitter

Beam flicker

spot shape

radiation damage

σ(N) = sqrt(N)

rms 11.5 e-/pixel

rms 0.57 ms

0.15 %/√Hz

pixels? mosaicity?

B/Gray?

signal vs noise

Which error dominates?

• Weak spots (high-res)background

• MAD/SAD (small differences)detector calibration

( if not rad dam! )

Background level sets needed photons/spot

Moukhametzianov et al. (2008). Acta Cryst. D 64, 158-166

Optimal exposure time(faint spots)

σtotal2 = σspot

2 + σbg2 + σreadout

2+ σraddam2

too long!

σtotal2 = σspot


2+ σraddam2

σtotal2 = σspot


2+ σraddam2


σtotal2 = σspot


2+ σraddam2

too short!

σtotal2 = σspot


2+ σraddam2


σtotal2 = Nphotons

+ σreadout2+ σraddam

2

needlessly long

Nphotons ≈ σdetector2


“optimal”

Nphotons ≈ 10x σdetector2


“buried”

Nspot + Nbg ≈ 10x m


“buried”

2

0

gain

0 + Nbg ≈ 10x m


“buried”

2

0

gain


2

00 10

gain

mgain

bgbghr


2

00 10

gain

mt

t

gain

bgbg

ref

hrref


0

2010

bgbggain

mtt

refrefhr

thr Optimal exposure time for data set (s)tref exposure time of reference image (s)bgref background level near weak spots on

reference image (ADU)bg0 ADC offset of detector (ADU)bghr optimal background level (via thr)σ0 rms read-out noise (ADU)gain ADU/photonm multiplicity of data set (including partials)

Short answer:

bghr ~ 100 ADU

for ADSC Q315r

What error dominates?

• Weak spots (high-res)background

• MAD/SAD (small differences)detector calibration

if not rad dam!

Optimal exposure time(anomalous differences)

I-I+

3%

100

phot

ons

10 photons

100

phot

ons


I-I+

3%

100

phot

ons

14 photons

100

phot

ons


3%

I-I+

2000

pho

tons

67 photons


1%

I-I+

20,0

00 p

hoto

ns

200 photons

Minimum required signal (MAD/SAD)

"#

)(3.1

fsites

DaMW

sd

I

“photon counting”

Read-out noise

Shutter jitter

Beam flicker

spot shape

radiation damage

σ(N) = sqrt(N)

rms 11.5 e-/pixel

rms 0.57 ms

0.15 %/√Hz

pixels? mosaicity?

B/Gray?

signal vs noise


σtotal2 = σspot


2+ σraddam2


0MADt

no detector is perfectly calibrated!

σtotal2 = Nspot + σbg

2 + σreadout

2+ σraddam2

+ (fshutterNspot )2 + (fflickerNspot )2

+ (fcalibNspot )2

σtotal2 = Nspot + σbg

2 + σreadout

2+ σraddam2

Fractional error

mult > (—)2Rmerge

<ΔF/F>

Damage model system

67 consecutive data sets

Data quality vs exposure

0.5

0.55

0.6

0.65

0.7

0 20 40 60 80 100

model vs warp

Exposure time (min)

Cor

rela

tion

coef

ficie

nt


579

1113151719212325

0 20 40 60 80 100

3A bin

Exposure time (min)


15

17

19

21

23

25

27

29

0 20 40 60 80 100

overall

Exposure time (min)

Data quality vs exposure1.7

1.9

2.1

2.3

2.5

2.7

0 20 40 60 80 100

I/sig(I) < 2

Exposure time (min)

Res

olut

ion

limit

Data quality vs exposure0

0.020.040.060.080.1

0.120.140.160.180.2

0 20 40 60 80 100

3A bin

Exposure time (min)

Rsy

m

Experimentally-phased map

Damage changes absorbance spectrum

0

500

1000

1500

2000

2500

3000

3500

4000

4500

50001

26

40

12

64

5

12

65

0

12

65

5

12

66

0

12

66

5

12

67

0

12

67

5

12

68

0

12

68

5

12

69

0

12

69

5

12

70

0

Photon energy (eV)

coun

ts


0

500

1000

1500

2000

2500

3000

3500

4000

4500

50001

26

40

12

64

5

12

65

0

12

65

5

12

66

0

12

66

5

12

67

0

12

67

5

12

68

0

12

68

5

12

69

0

12

69

5

12

70

0

beforebeforeburntburnt

Photon energy (eV)

coun

ts


0

500

1000

1500

2000

2500

3000

3500

4000

4500

50001

26

40

12

64

5

12

65

0

12

65

5

12

66

0

12

66

5

12

67

0

12

67

5

12

68

0

12

68

5

12

69

0

12

69

5

12

70

0

beforebeforeburntburnt

Photon energy (eV)

coun

ts

1

0

fluorescence probe for damage

Absorbed Dose (MGy)

Fra

ctio

n u

nco

nve

rted

Wide range of decay rates seen

0.

0

0

.2

0.4

0.6

0

.8

1.0

0 50 100 150 200

Half-dose = 41.7 ± 4 MGy“GCN4” in crystal

Half-dose = 5.5 ± 0.6 MGy8 mM SeMet in NaOH

Protection factor: 660% ± 94%

http://ucxray.berkeley.edu/~jamesh/elves

Download Elves from:

powerpoint file available: ~jamesh/powerpoint/ oslo_2010.ppt

Documents

user input distance

adsc q4 detector

datasemetreport elves

images andset

pptht elves

office of science

advanced light sourceals

materials sciences division