shot rate improvement strive for the …icalepcs.synchrotron.org.au/talks/mod3o03_talk.pdf ·...

LLNL-PRES-677128

This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC

SHOT RATE IMPROVEMENT STRIVE FOR THE

NATIONAL IGNITION FACILITY (NIF)

15th International Conference on Accelerator & Large

Experimental Physics Control Systems (ICALEPCS)

October 18-23, 2015

Gordon BruntonNIF Control Systems Lead

MOD3O03

Lawrence Livermore National Laboratory2

Shot Rate Goals:

• FY15: 300 shots (>50%

increase over FY14)

• FY16: 400 shots

Primary Focus Areas:

• More shot time

• Improved experiment

scheduling

• Reducing shot to shot

durations

With NIF fully operational as a user facility focus now

shifted to maximizing return of experimental data

With changes already implemented to formalize 24/5 shot time and improved

experiment scheduling focus shifted to reducing shot cycle durations

Focus Areas for Shot-Rate

Improvement

Planned

ICALEPCS 2015 - MOD3O03


The NIF Control System is one of the world’s largest

operational scientific control systems

Large scale

• 66,000 device control points

• >1M I/O channels

Highly data-driven

• Device configuration

• Experiment definitions, model & results

Highly distributed

• 35 Framework & Supervisory servers

• 3 compute clusters (110 nodes)

• 950 Front-End Processors

• 900 embedded controllers

• 2,400 processes

Highly automated

• 1.6M sequenced control point operations per shot cycle

• 24x7 operation

Automatic control of 192-beam shots

is overseen by 14 operator stations

NIF Control Room

A large scale systems analysis and engineering effort was performed to identify

where to best invest in controls enhancements to increase the NIF shot rate



Years of shot cycle metrics analyzed to determine ‘normal’ critical paths

• Shot cycles categorized by configuration complexity (i.e. warm simple,

warm complex, cold and layered) due to high variance in execution times

Shot cycle sequences analyzed to identify if ordering changes could reduce

critical path durations (top down)

Long duration shot cycle sequences analyzed in depth for optimization

and/or elimination (bottom up)

• Analysis considered both critical path and ‘close to’ critical path activities

to ensure true return of investment was measured

Improvement activities were prioritized based on assessed time savings

and implementation effort

Several analysis approaches were used to identify the

largest savings

The following summarizes some of the key improvements chosen and results achieved



* Shot Block Durations based on averages of most recent 3 month shots

ISP Rod Shots

Align KDP/Main

Laser

Lase

r P

rep

Sh

ot

Load

Shot

Target Alignment

TD Dry Runs

Pinhole

ChecksOSP

Rod TAS

OutDIM

Final

System Shot

NIF

Sh

ot

Load

ShotRegister

TCCDIM Alignment Initial

TAS

AlignBeams

Pointing

Duration (Hrs)1 2 3 4 5 6

Performing alignment & energetics calibration in parallel

with target area operations reduced shot cycle by 1 hour

Utilizing new ‘slack’ time significantly reduced the number of independent

calibration and verification shot cycles previously required

New Parallel Shot Cycles

Perform

energetic

qualification off

critical path

Savings of 1

hour per shot

possible

Creates

‘slack’ for

other

activities


Amplitude modulation

verification post

experiment wavelength

change

• Operational cost* =

120 * 2hrs per year

Precision pulse shape

calibration for high

precision experiments

• Operational cost * =

150 * 3hrs per year

AM Modulation

Pulse Shape

Other activities utilized ‘Slack’ Time

* Estimates based on historical shot metrics extrapolated forward to nominal goal rate of 400 shots per year


Target alignment process analyzed to automate some

operations and minimize operator interactions

Use of TAAT for target alignment has significantly reduced the duration & variance of

alignment times and is now under evaluation for diagnostic alignments

Target Alignment Assistant Tool (TAAT) was developed to provide

graphical scripted interface to guide operator through fine alignment

process for NIF positioners

Semi-automates manual alignment approach by removing opportunities

for user input error

• Data driven approach to allow ease of adaptability to novel target types

Shot cycle savings of 30-60 minutes obtainedTAAT tool in-use

Operational savings using TAAT

#

Measurements

#

Moves

# Data

Entries

# Move

Choices

#

Measurements

#

Moves

# Data

Entries

# Move

Choices

Total 1413 130 83 26 763 34 0 0

Savings (%) 46% 74% 100% 100%

Manual Alignment

ProcedureTAAT



Target chamber positioner movement rules of

engagement re-evaluated to reduce bottlenecks

Alignment duration variances have significantly reduced with new rules without

compromising safety while allowing greater operational flexibility

All target chamber positioner movements previously required 2 operators,

for safety, which has caused delays due to staff availability and distraction

New rules of engagement relax requirement to 1 operator for 90% of

moves without compromising safety

My Chamber Rule Solo Positioner Rule Retract Mode Rule



* Shot Block Durations based on averages of most recent 3 month shots

ISP Rod Shots

Align KDP/Main

Laser

Lase

r P

rep

Sh

ot

Load

Shot

Target Alignment

TD Dry Runs

Pinhole

ChecksOSP

Rod TAS

OutDIM

Final

System Shot

NIF

Sh

ot

Load

ShotRegister

TCCDIM Alignment Initial

TAS

AlignBeams

Pointing

Duration (Hrs)1 2 3 4 5 6

New ‘Gatling’ experiment type facilitates back-to-back

target shots in the same shot cycle

The first series of ‘Gatling’ experiments are scheduled to be performed on NIF this fall

and have the potential to significantly contribute to further shot rate improvements

Shot 1

Avoids unnecessary laser preparation if shot configurations are similar

Only target exchanges and diagnostic reconfigurations between shots

• Can utilize interleaved positioners for further savings

Shot 2 .. n

3 Hr Turnaround

Shot Complete

DIM

ParkOptional, only performed if entanglable

TARPOS

Retract

TAS

insert

TAS

Out

DIM

Insert

And Repeat

6 7 8 9

System Shot

NIF

Sh

ot

n

Target Exchange

Target Align

6 Hr Turnaround



Sequencing of optics inspection (FODI) analyzed and

optimized to minimize shot to shot turnaround

With limited archiving in this area, optimization analysis made significant use of SplunkTM

log parsing ability to identify optimization strategies and evaluate return on investment

FODI Steps w/ Proposed OptimizationsDuration per

beamline

(seconds)

Savings per

beamline

(seconds)

Laser OFF

Laser Switch to Beamline

Focus Motor to Position (Brake Off & Brake On)

Laser On

Capture and Archive Image

Image Capture Complete

Laser OFF


Focus Motor to Position (Brake Off & Brake On)

Laser On



Laser OFF


Focus Motor to Position (Deferred Braking)

Laser On



Laser OFF


Focus Motor to Position (Deferred Braking)

Laser On



Duration (Seconds)

With acquisition script optimizations

50 23%

Overall savings

Present FODI acquisition

65 0%

With 'Deferred Braking' optimization

40 38%

38 42%

Optimization Results

FODI Inspections performed 2-4

times per week between shots

- Savings = ~110 Hrs per year



To assist with measuring efficiency improvements a

critical path analysis tool (CPAT) has been developed

CPAT already used to identify 5 minute saving from sub-optimal rod shot sequencing.

NIF performs ~1300 rod shots per year (108 operational hours).

These savings equate to ~11 additional shots per year

Critical Path Analysis Tool (CPAT) Analyzes historical shot

cycle metrics

Visualizes critical path of

shot cycle(s)

Provides metrics on shot

execution including long

operations, averages

and standard deviation

Rapidly identifies ‘slack’

time on non-critical path

blocks



The results of the shot cycle schedule and controls

enhancements have resulted in significant improvements

FY15 Weekly Shot RateFY15 Target Shots

The shot rate process improvements resulted in meeting the FY15 300 shot rate goal

>1 month earlier than planned

FY15 goal of

300 shots

met 8/14/15

~80% increase

over FY14



Advanced Tracking Laser Alignment System (ATLAS)

• Laser Tracker based diagnostic package alignment

• Replaces need for opposed port imaging systems

• Decouples diagnostic alignment from use of Target

Alignment System (TAS) thus removing diagnostic

alignments from the shot cycle critical path

Target And Diagnostic Manipulators (TANDM)

• Addition of 2 new target/diagnostic positioners

• Allows additional diagnostics and allows Cryo positioner to be layering

without impacting shot schedule

• Requires ATLAS as no opposing port alignment system (OPAS) being

implemented for alignment

Future Work

FARO Laser Tracker



Historical metrics were critical to process improvement

• Invaluable in accurately analyzing optimization approaches and

measuring success

Both top down and bottoms up analysis approach identified improvements

• Top down typically yielded the most gains (i.e. big picture)

Return on investment important to accurately capture

• Aids in defining need for change and prioritizing order of deployment

Reliability, Availability & Maintainability (RAM) also important to analyze

• With parallel execution the slowest cog governs speed of overall system

System optimizations often best left until system is completed

• However imposed system constraints should be considered throughout

design to ensure optimization potentials are not being inhibited

Summary

NIF shot rate has made significant gains during FY15 and optimizations implemented

have positioned us well for meeting FY16 goals and beyond


shot rate improvement strive for the …icalepcs.synchrotron.org.au/talks/mod3o03_talk.pdf ·...

Documents