The ATLAS IBL CO2 Cooling

System

International Workshop on Semiconductor

Pixel Detectors for Particles and Imaging

(Pixel 2016)

(Sestri Levante, 5-9 September 2016)

On behalf of the ATLAS collaborationVERLAAT, Bart (CERN), OSTREGA, Maciej (CERN), ZWALINSKI, Lukasz (CERN), BORTOLIN, Claudio

(CERN), VOGT, Sven (MPI-Münich (DE)), GODLEWSKI, Jan (Polish Academy of Sciences (PL)), CRESPO-LOPEZ, Olivier (CERN), VAN OVERBEEK, Martijn (Nikhef), BLASZCZYK, Tomasz (CERN)

107/09/2016 B. Verlaat

ATLAS IBL: A new 1st layer

around a reduced beam pipe

2

IBL pixel sensors

IBL Carbon stave with cooling pipe

IBL Stave

IBL an extra pixel layer

Component

(32 per stave)

Power

(W/unit)

Power

(W/stave)FEI4 chip 1.12 35.84Pixel sensor (after irradiation) 0.68 21.61Stave flex 0.17 5.38Type 1 cables 0.17 5.38

Total per stave 68.21Total for 14 staves 954.94

Cooling temperature required: <-35°C

Connectiontube bundle14x 3x0.5mm

ID end plate dry volume

*

**

** *

**** * ****

* *

* **

* * * *** * **

**

*

Inner detector (TRT+SCT+Pixel)

Electromagnetic calorimeter (LAR)

Hadronic calorimeter (Tile)

Solenoid magnet

Splitter box (Concentric split and vacuumtermination)

Vacuum insulated concentric tubes (flex lines)(7x1.6x0.3mm inlet inside 4x0.5mm outlet)

Manifold box

Junction box

Vacuum insulated transfer lineMuon area sector 5

(LAR)

Beam pipe

IBL detector

The IBL cooling loops

3Manifold box in S5

PP1 connectors

Splitter box in IDEP

1 round trip = 1 loop = 31m

Flex line routing in IDEP

Evaporator tubes in staves 1.5mm

7 miniature vacuum insulated flexible cooling lines (<18mm)

Cooling plants in USA-15

4

IBL has 2 redundant CO2 systems, in case of failure it swaps transparent to the other system. Detector is minimal affected (small temperature fluctuation)

System ASystem BControls

Accumulator

Front view

Rear view

Tran

sfer line

CO2 plant commissioning

• The cooling system is commissioned over a by-pass dummy load (3kW) and with the detector attached

• The cooling system is very stable (<<1’C) over a large temperature range (from room temperature down to -35°C

Cooling plant Junction box with dummy load

11

8

9

USA15 UX15

Manifold box with detector loops

1

Liquid flow2-phase flow

2

5

5”

6

7

3 4

7”

10

-35°C set-point

20°C coolingAccumulator pressure lowering for cool down

Pressure increase for liquefying prior to start-up

Sub cooled pumped liquid( follows chiller temperature)

Saturation temperature

Liquid cooling

2-phase cooling

11

2

3, 5”, 7”(Tsat)

9 Sub cooled liquid margin for pump operation (>10 °C)

0 20 40 60 80 100 120 140 160 180-30

-20

-10

0

10

20

30

Days since 23 June 2014

Tem

pera

ture

Stave1 temperature

“Warm” commissioning

Bake-out

1st cool cold operation

Steady state tests

Detector temperature during commissioning

Junction box cooling tests (Detector by-passed)

En

d o

f co

mm

issi

on

ing

June 14 Dec. 14Simplified system schematics

Operational temperature range

8 hours

20

0

-20

-40

-60

Beam pipe bake-out• The beam pipe bake-out was a crucial event for the IBL

cooling

• The beam-pipe was heated in steps to 230⁰C, the cooling system prevented the IBL for overheating

• A 2 fault redundant operation of the cooling was established o 2 systems operated in parallel

o System back-up by a bottle battery blow system

• Results and observationso Maximum recorded sensor temperature was -8⁰C @ 230⁰C beam

pipe

o Due to twice the flow (2 systems in parallel) => ca. 4x pressure drop, most staves stayed single phase for long time.

• No serious issues during bake-out, system run without problems

6

-25.0

-20.0

-15.0

-10.0

-5.0

-50.0 0.0 50.0 100.0 150.0 200.0 250.0

Co

olin

g t

emp

erat

ure

s (°

C)

Beam pipe temperature ('C)

IBL Cooling temperatures at bake -out

-Stave01 InletStave01 OutletStave01 ModulesStave05 InletStave05 OutletStave05 ModulesStave07 InletStave07 OutletStave07 Modules Full boiling

Partly boilingLiquid cooling Stave 14 behavior

630 640 650 660 670 680 690

-20

-10

0

10

20

30

40

Time(min)

Flo

w (

g/s

), T

em

pera

ture

(`C

)

IBL-cooling blow off test with 3kW heating

FT901 Blow system flow

FT106 IBL-A flow

FT306 IBL-B flow

TT116 Junction box liquid

TT136 Junction box return (2phase)

TT117 Heater

Failure of plant B

Delivered flow

Failure of plant A Cooling temperature remains stable

Heater temperature

25 minutes of stable 3kW cooling using blow system (3kW Test was stopped to save CO2)

Blow

Sys. BSys. A

The IBL cooling heat loads

• The cooling system has to coop with several heat loads:o Detector electronics power

o Ambient heat leak in the detector

o Ambient heat leak of the system

• The detector power now is about ~30 Watt/stave (420 W total)

• The total absorbed heat load can be measured when both in and outlet of the plant are in liquid phase (occasionally present)

• The ambient heat leak in the detector, is the same order as the electrical power

• Total load for the cooling system (detector + ambient)o Current operation: 1.5 kW at low temperature

o Expected EOL: 2 kW at low temperature

o Tests have been done with 3kW dummy load

7

Total ambient heat leak

Detector power during measurements (dec 14)

Component

(32 per stave)

Power

(W/unit)

Power

(W/stave)FEI4 chip 1.12 35.84Pixel sensor (after irradiation)

0.68 21.61

Stave flex 0.17 5.38Type 1 cables 0.17 5.38

Total per stave 68.21Total for 14 staves

954.94

-35 -30 -25 -20 -15 -10 -5 0 5 10 150

100

200

300

400

500

600

700

800

900

1000

Accu setpoint temperature (°C)

Hea

t lo

ad

(W

att

)

IBL Cooling system heat loadsMeasured ambient heat (IBL=on)

Measured ambient heat (IBL=off)

CoBra ambient heat (IBL=on)

CoBra ambient heat (IBL=off)

IBL Ambient heatload fit

Total Ambient heatload

Total ambient heatload fit

IBL Detecor heat load

IBL Detector heatl load fit

Detector power

From slide 2

End of commissioning steady state

tests and comparison with simulations

(Dec. 2014)

8

0 5 10 15 20 25 30

-30

-28

-26

-24

-22

-20

-18

Ju

nctio

n B

ox

Ma

nifo

ld B

ox

Flex line IBL v olume

Inlet:

Cable temperature: -13.6±0.8 ºC / -11.2±0.9 ºC

Cable board temperature: -16.5±0.3 ºC / -10.3±0.3 ºC

Outlet:

Cable temp.: -13.6±1.4 ºC / -11.3±1.6 ºC

Cable board temp.: -16.5±0.4 ºC / -10.4±0.5 ºC

HeatLoads:

Ambient (CoBra): 446.6 W / 579.6 W

Ambient (Measured): NaN W / NaN W

Modules: 0 W / 355.4 W

Miscalaneous:

Pressure drop: 5.3 Bar / 5.5 Bar

Massf low: 18.1 g/s / 17.9 g/s

Ambient temp.: -16.5 ºC / -10.3 ºC

Exit v apor quality : 0.08 - / 0.17 -

Distance from Junction Box (m)

Tem

pera

ture

(ºC

)

IBL temperatures for set point -30ºC; Measured data and CoBra simulation results

0 1 2 3 4 5 6 7 8 9

-30

-28

-26

-24

-22

-20

-18

IBL position (m)

Tem

pera

ture

(ºC

)

dTTFoM

=3.6 ºC

dTHTC

=1.7 ºC

dTdP

=3.2 ºC

Set point=-30 ºC

TModules

=-21.5 ºC

dT

tot =

8.5

ºC

Detailed view of IBL region

CP temperatures (IBL pow er=off)

Saturation temperatures (IBL pow er=off)

Module temperatures (IBL pow er=off)

CoBra Saturation line (IBL pow er=off)

CoBra Fluid temperature (IBL pow er=off)

CoBra structure temperature (IBL pow er=off)

CoBra w all temperature (IBL pow er=off)

CP temperatures (IBL pow er=on)

Saturation temperatures (IBL pow er=on)

Module temperatures (IBL pow er=on)

CoBra Saturation line (IBL pow er=on)

CoBra Fluid temperature (IBL pow er=on)

CoBra structure temperature (IBL pow er=on)

CoBra w all temperature (IBL pow er=on)

Manifold and junction box

The IBL

Additional result numbers

Set point = -30°CAmbient heat load from previous slide included in simulations

Steady state tests (Dec. 2014)Overview of different set point temperatures

9

-35 -30 -25 -20 -15 -10 -5 0 5 10 15

0

1

2

3

4

5

6

7

8

9

10

accumultor set point temperature (°C)

IBL T

em

pera

ture

gra

die

nts

(°C

)Steady state detector temperature offsets from setpoint

Module 1 det off

Module 1 det on

Module 2 det off

Module 2 det on

Module 3 det off

Module 3 det on

Module det 4 off

Module det 4 on

Module det 5 off

Module det 5 on

Module det 6 off

Module det 6 on

Module det 7 off

Module det 7 on

Module det 8 off

Module det 8 on

JB return det off

JB return det on

MB return det off

MB return det on

CP outlet det off

CP outlet det on

CP inlet det off

CP inlet det on

MB liquid det off

MB liquid det on

JB liquid det off

JB liquid det on

Cooling pipe temperatures

Gradients increase with colder cooling temperature

Gradients due to stave conduction relatively constant

First modules show a larger and more irregular temperature offset (boiling onset issue)

Data of previous slide

Cooling performance during

operation

1007/09/2016 B. Verlaat

The cooling system is stable within 0.2 °C (<0.05 °C RMS), Detector modules within a few degrees due to power fluctuations

Cooling pipe measured stability during M9 run

Module and cooling pipe temperatures

Nick Dann

28/0

8/16

01/0

5/16

Antonio Miucci

2015**

** Please note that the shown temperatures in 2015 show values which are not yet corrected with the calibration constants defined in April 2015. As a consequence cooling pipe sensors shown are ˜1.5°C too high, Module temperatures ˜0.5°c

Operational experience since LS1

(1.5 years of operation)

• The IBL CO2 cooling system had an excellent track record: 0% downtime during physics

• Only limited interventions were needed, of which most were done during safe periods (TS, MD)

• 2 incidents during detector operationo 1 hardware failure with a successful back-up procedure

o 1 DSS interlock due to a threshold conflict in the stepper introduced for long term high temperature operation, interlock occurred during a controlled intervention

• We are getting positive feedback of the system operation and reliability by ATLAS

• Very good cooperation between the operator team (ATLAS+EN-CV) and development team (EP-DT)

1107/09/2016 B. Verlaat

Period CO2 System failure Chiller or primary cooling issue

Software modification

System test Plant recovery

DSS Interlock

Automaticswap

Maintenance intervention

Data taking

Beam 1x (Flow meter failure)

4x 5x 1x

No beam

1x (Initiated swap for maintenance followed by a trip of the 2nd system due to threshold conflict in the start-up stepper)

6x 1x

TS 5x 6x 1x 1x

MD 1x 1x 3x 3x 1x

Boiling onset problems

1207/09/2016 B. Verlaat

• Sometimes the boiling is not triggered, and a reduced cooling performance is observed (bad heat transfer)

• We are trying to understand what causes this phenomena and how we can improve it

• The non-boiling issue is annoying for the alignment due to stave bowing.

• A test set-up containing a real size stave pair including flex lines and its orientation is made in SR1 to investigate the phenomena

SR

1 L

evel

-1

SR

1 cl

ean

ro

om

DAQ rack

PC

Manifold box

Vacuum station

CO2 plant

Flex lines with support

Dummy staves

Rep

rese

nta

tiv

e h

eig

ht

dif

fere

nce

**** * ***

**

*

+ - Direct heating on pipe

Heating with silicon heaters on real stave

2657mm1412mm

2657mm

D2.2x0.1mm

D2.2x0.1mm

D1.7x0.1mm

D1.7x0.1mm D1.7x0.1mmCooling stationin SR1

Manifold box section

Spare flex lines lines

Cooling pipe

Modules

Power

1st results from SR1 test

• 1st results from SR1 show the superheated liquid in the inlet tube

• Flow reduction looks promising

• Flow reduction needs a modification in the manifold to avoid manifold evaporationo Will be tried in SR1

1307/09/2016 B. Verlaat

FlexlineIDEP

Flow direction

Inlet cooling pipe

PP

0

PP

0

PP

1

PP

1

Inlet cooling pipe

IDEPFlexline

IBL

Super heated liquid present in inlet pipe

Boiling triggered

Krzysztof Sliwa, 31-08-’16

Summary and conclusions

• The IBL CO2 cooling system was successfully commissioned in 2014.

• The cooling system has been fully operational since LS1 for 1.5 years and has performed very reliable with 0% downtime for ATLAS physics.

• The cooling system wide operational temperature range has proven to be very convenient to solve the LV-current issue.

• The high temperature stability of the cooling is very advantage for the stave bowing issue.

• The boiling onset issue is studied in a real size cooling loop mock-up in SR1.

• The IBL cooling method and operational experience has proven to be a good baseline for the ITk cooling.

1407/09/2016 B. Verlaat