![Page 1: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/1.jpg)
GL
Progress towards connecting
Global Phasing’s workflow
with MXCuBE
Gerard Bricogne, Rasmus Fogh, Global Phasing
MXCuBE meeting, GrenobleJanuary 2017
![Page 2: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/2.jpg)
GL
• Preamble
• Workflow architecture
and MXCuBE connection
• Workflow demonstration
GL
Contents
![Page 3: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/3.jpg)
GL
Introducing myself
• Coming from NMR, CCPN project
– Data models and standards
– Python application development
– Software integration
– Structural biology
• I am not a crystallographer
– but I can learn.
– Please be patient
![Page 4: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/4.jpg)
GL
• Preamble
• Workflow architecture
and MXCuBE connection
• Workflow demonstration
GL
Contents
![Page 5: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/5.jpg)
GL
Workflow operation
• Drives the beamline, data collection and
(eventually) processing
• Beamline-independent core; messages
using Abstract Beamline Interface
• Works through beamline control system
• Input, instructions and output through
beamline user interface (or emulator)
![Page 6: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/6.jpg)
GL
Architecture
Workflow
Datapersistence
JavaXDS
SimcalStratcal
…
Fortran
Wolfgang Kabsch,Wlodek
Paciorek,Global
Phasing…
GDAbeamlinecontrol
Java
Diamond
MXCuBEbeamlinecontrol
MXCuBE
Python
Emulation
TESTEDlive on
DLS-I04 !
Peter Keller
Messages
JMS messagebroker,
ActiveMQ
Py4J,Python
messages
New (me)
DEMO
![Page 7: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/7.jpg)
GL
MXCuBE messaging
• Synchronous RPC calls from workflow
– Beamline side py4j server
– Java message objects
– Python access via py4j
– Conversion to pure Python message objects
matching Abstract Beamline Interface
– Self-contained, beamline-independent,
can be replaced
• Messages trigger processMessage calls
– Hook for beamline emulator, or beamline control
– Intended to hook into MXCuBE queue
![Page 8: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/8.jpg)
GL
Messages from Workflow
• RequestConfiguration
• RequestCentring
• ChooseLattice
• GeometricStrategy (requests re-centring)
• CollectionProposal (requests data)
• Information messages:– SubprocessStarted
– SubprocessStopped
– Text (for echoing)
• Workflow Ready/ Aborted / Completed / Failed
![Page 9: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/9.jpg)
GL
Messages to Workflow
• ConfigurationData
• PriorInformation (Sample initial data)
• SampleCentred
• SelectedLattice
• CollectionDone
• BeamlineAbort
• CentringDone (for calibration workflow)
![Page 10: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/10.jpg)
GL
Data Collection UML
![Page 11: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/11.jpg)
GL
Instrument Setting UML
![Page 12: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/12.jpg)
GL
Future issues
• Messaging protocol– Synchronous or asynchronous?
– Is py4J OK?
– XML-RPC server?
• Handshaking and control– Who triggers whom?
– Processes, aborts, and restarts
• User interface– Need for changes or dedicated modules?
• Calibration and configuration– High standard required
– Bespoke calibration workflows
![Page 13: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/13.jpg)
GL
• Preamble
• Workflow architecture
and MXCuBE connection
• Workflow demonstration
GL
Contents
![Page 14: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/14.jpg)
GL
Demonstration
• I show running demo, interspersed with screenshots from live DLS-I04 test
• Workflow program runs in full production mode
• Client-server communication fully functional
• Beamline control system (BCS) is emulated by a plug-in replacement (mock-BCS)
• Mock-BCS runs as separate server process
![Page 15: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/15.jpg)
GL
Beamline emulation
• Mock-BCS based on
– Configuration files
– Sample description files
– Dummy centring values
– User input
– Copies of pre-acquired characterisation images
• Full emulation (existing, not demonstrated)
– Images calculated from structure model,
mosaicity value, specific crystal orientation, …
![Page 16: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/16.jpg)
GL
Workflow demo 1
• Start mock-BCS and workflow
• Load sample information and parameters
• Centre sample
![Page 17: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/17.jpg)
GL
DLS-I04 live - image 1
![Page 18: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/18.jpg)
GL
Workflow demo 2
• Collect characterisation data
• Run indexing (XDS, COLSPOT, IDXREF)
• Select indexing solution
![Page 19: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/19.jpg)
GL
DLS-I04 live - image 2
![Page 20: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/20.jpg)
GL
Workflow demo 3
• Calculate spot predictions from indexing
solution for characterisation images
(XDS, simcal_predict)
• User check of indexing solution (GPX2)
![Page 21: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/21.jpg)
GL
DLS-I04 live – image 3
GPX2 display of characterisation images
This example shows some mismatch between predictions (red ellipses) and measured spots, probably due to a lack of up-to-date calibration data for the goniostat.
The spots used for indexing (faint blue crosses) come from the SPOTS.XDS file. Since these characterisation images are fine-sliced (0.1⁰) the spots are all partial, so that many of their blue crosses will be found on adjacent images.
![Page 22: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/22.jpg)
GL
Workflow demo 4
• Calculate acquisition strategy (stratcal)
![Page 23: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/23.jpg)
GL
DLS-I04 live - image 4
![Page 24: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/24.jpg)
GL
Workflow demo 5
• Check sample re-centring for each sweep
of stratcal acquisition strategy
![Page 25: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/25.jpg)
GL
DLS-I04 live - image 5
![Page 26: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/26.jpg)
GL
Workflow demo 5
• Collect data
• Demo ends here
![Page 27: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/27.jpg)
GL
Where we are
• Workflow engine fully operational
– Tested and working on live beamline
(GDA/ASTRA, DLS-I04)
• Message transfer to Python fully operational
– Running under demo
– Current (prototype) implementation could be
replaced
• Next step is connecting to MXCuBE queue
![Page 28: Progress towards connectingmxcube.github.io/mxcube/doc/past_meetings/2017_esrf/MXCuBE-Gr… · Workflow demo 3 •Calculate spot predictions from indexing solution for characterisation](https://reader033.vdocuments.us/reader033/viewer/2022051923/60110c951d766c384e783f25/html5/thumbnails/28.jpg)
GL
END
GL