ICS re-planning, outsourcing/inhouse development strategy and staffing

TAC 12 presentation

Henrik Carling

ESS/ICSDate: 2015-09-30

2

Re-planning, outsourcing/inhouse development strategy and staffing• Agenda

– Current situation• Planning -> Effort estimate -> Competence/capacity roadmap• The challenge of in-kinding labor• Management of commercial suppliers

– Issues

– Way forward

3

2016 plan/budget result

• A fairly detailed plan/budget for 2016 is available

• For some workpackages, it is a major revision of the planning. The most common problem is the low availability of stakeholder requirements meaning uncertain scope

• The plans for 2017 and onwards are (with few exceptions) untouched since 2013/2014 timeframe. This means that visibility for activities after 2017 are obscure

• Plans for 2016 show that we must produce about twice the value that will be done in 2015

• For 2017 another value doubling is planned

• The ICS budget envelope for the construction phase shows the monumental challenge of fast rise and fall in capacity over just a few years time

0

2000000

4000000

6000000

8000000

10000000

12000000

14000000

Total

0

5000000

10000000

15000000

20000000

25000000

2015 2016 2017 2018 2019

Physics

Integration - Target

Integration - Conventional facilities

Test Stands

Integration - Instruments

Personal safety system

Control system infrastructure

Software Applications

Machine protection system

Management and administration

Integration - Accelerator

Hardware core

Software core

Equipment

4

Employee/consultant ratio

• Since it is not currently possible to establish a credible competence / capacity roadmap, some other models can be used to establish the relationship between employees and consultants

• One important factor is the size of the ICS team at steady state operations (somewhere in the phase 2019 - 2026)

• Based on experience from other facilities and adjusting for ESS Size and complexity, the following model has been used as an estimate for the steady state operations phase

0

10

20

30

40

50

60

70

2015 2016 2017 2018 2019

ICS staffing models (constant budget)

Consultants

Employees

Group EmployeesInfrastructure 4Integration 7Management 3Safety and protection 7Software 9

5

The challenge of in-kinding labor

• The in-kind budget model is particularly difficult for ICS since 80 - 90% of the cost is labor driven• This means that approximately a value of 25 M€ shall be generated from In-kind activities• This corresponds to approximately 231 MY with the ESS Cost-book labor rate• Meaning that approximately 58 in-kind FTE:s shall be fully loaded by ICS activities during 2016-2019• This is very, very difficult

• We expect the ”value efficiency” to be quite low for in-kind activities

Organization Title Estimate Agreed Project status Contractual statusBilbao MEBT controls 570Bilbao MEBT controls 130Estonia TUT Electronics collaboration Early discussions to identify opportunities in testingFrance CNRS Integration Support WP.10 600Huddersfield Control Boxes 2000 Activities postponed to 2H 2016IFE Control Room design 1100 Pre-study made and delivered. Pending contracts Technical annex being drafted. HOA being discussed assisted by Marie-LouiseLegnaro DTL controls 500PSI Electronics collaboration 700 Collaboration is ongoing - investigation started how to In-kind this workSaclay LEBT controls 260 260 Technical annex signedSaclay Proton Source 390 390 Technical annex signedSaclay Proton Source and LEBT 150 150 Technical annex signedSaclay RFQ ontrols 600UK STFC Procurement for PSS 1850 Activities postponed to 2H 2016Uppsala Control System 175 Activities postponed to 2016 - coordinate with AcceleratorUppsala Control System 150 Activities postponed to 2016 - coordinate with AcceleratorUppsala Control System 100 Activities postponed to 2016 - coordinate with AcceleratorZHAW MPS interlock Collaboration is ongoing - investigation started how to In-kind this work

6

Management of commercial suppliers

• Competence is one of several important factors for line managers to consider

• Social skills and overall team and motivation level are sometimes more important (since skills often can be learned more quickly than behavioural changes)

• When extending the line organization, one of the first decision points is if the position is permanent or transient

• Permanent positions should be filled with employees considering– Long term (5 - 10 years) assignment, competence and capacity profile of the team– Social/teamwork skills– Competence and experience– Remuneration and other contractual status

• Transient positions can be filled with temporary employees or consultants– New framework agreements are being created

• Software development services• Hardware development services• PLC programming services • Integration services

• The new framework agreements are intended to replace the current ”Epics integration” agreement– Increase supplier selection– Provide local support for smaller assignments– Increase supplier competitiveness– Lower cost, increase quality

7

Technology scope for ICS

• The span of technical scope and the competencies needed in ICS is enormous• From analog signals to Java-based high-end, big-data GUI:s in high performance, high availability• PLC systems• Mechanics and Safety/protection systems• Systems integration, Installation• Project management and administration

Electronics hardware Software

PLC

Mechanics

Anal

og e

lect

roni

cs

Digi

tal e

lect

roni

cs

FPGA

firm

war

e

Digi

tal c

omm

unica

tion

PCB,

pac

kagi

ng

Test

and

val

idati

on

Indu

stria

lizati

on, m

aint

enan

ce

Kern

el d

river

s

Ope

ratin

g sy

stem

s (RT

)

Deve

lopm

ent e

nviro

nmen

ts

EPIC

S

CSS/

Eclip

se

Appl

icat

oins

(GU

I, W

eb, S

ervi

ces)

Data

base

s

Avai

labi

lity,

perfo

rman

ce

Test

and

val

idati

on

Safety and protection

8

Competence map for ICS

• Electronics (Test, general)• Digital systems (FPGA, Firmware)• Automation and control (PLC)• Computer communications (Networking, cabling)• Hardware/software interaction (Low level software)• Core software development (Linux kernel)• Software development (Java)• Database, High availability (Postgres)• Systems integration• Safety and protection systems (IEC61508 etc)

9

Software development services framework

RequirementsThe European Spallation Source ESS AB is an accelerator-driven neutron spallation source. The Integrated Controls Systems (ICS) is responsible for providing control and monitoring for all parts of the machine (accelerator, target, neutron scattering systems and conventional facilities).

A large part of the construction scope for ICS is to design, develop, test, deploy and maintain software systems. The software for the control system of ESS is distributed over many different products and technologies with a very wide scope. Some examples are:

• Control system core functions; the ESS software control system will be based on the open-source standard EPICS [Link] family of projects

• Hardware drivers; specialized hardware built specifically for ESS need software drivers, most often drivers in the Linux kernelspace

• Database oriented systems for configuration management; most user level ESS projects are built on database technology, common examples use PostgreSQL or similar open database products

• Operator environments; Providing ESS operators with a user interface environment for the control system is essential for ICS. We will use the Control System Studio [Link] family of open-source, Eclipse-based products for this purpose

• Development environments; in order for ESS stakeholders to be able to integrate efficiently into the EPICS-based control system, ICS will provide ready-made development environments to facilitate integration work

• Data acquisition, archiving and analysis; ICS needs be able to register and store signal flow in the control system to be able to debug and analyze/optimize problems in the very complex facility

10

Software development services framework

This “OCT” has been issued by ESS AB in order to enter into a Framework Agreement with one supplier, and for a maximum period of 4 years from contract signature, for the following products:• Software development services including the following

– Design• Requirement breakdown and analysis• Estimations• Software architecture, data structure

– Implementation• Analysis• Documentation• Programming

– Java, C, C++– Linux environments, Embedded environments– Test driven development

• Database implementation– Test, validation, verification and integration

• Several different levels (module, system, integration, …)

• Build systems, automated testing• Continuous integration• Delivery

– Deployment, maintenance and support

• Software methodology services including the following– Software project management support

• Many methodologies from agile to waterfall• Planning and structuring software projects• Coordination of production/deliveries from several

sites• Software architecture• Coaching and training

– Useability and design support• User interface design analyses• User experience design support

The following products are excluded:• Automation software development (PLC)

11

Issues

• Very difficult to achieve the in-kind goals as labor

• Long lead time for framework agreements to come into play

• Uncertainties around the steady-state operations staffing situation

12

Thank you!

• Questions