london bridge point failures investigation report bridge point failures investigation report draft 7...

London Bridge Point Failures Investigation Report

Draft 7 – NOT FOR CIRCULATION

of 11

EXECUTIVE SUMMARY:

Following the August Thameslink blockade, four point failures occurred in the London Bridge Station area from

31/08/16 to 07/09/16, each resulting in significant delays.

The four points that failed were part of eight critical point ends installed in Christmas 2015 in preparation for the

opening of platforms 7 – 9 in August 2016 and the introduction of the new Charing Cross timetable. Prior to August

Charing Cross services were not stopping at London Bridge and the points were only in occasional use. Through

readiness reviews it was recognised that the points would be under increased usage post August and testing and

health checks were undertaken in preparation for this.

An investigation has been carried out to understand the root cause of the failures and has identified the following:

Three of the four failures were due to the incorrect detailed set-up and adjustment of the points

One failure was caused by a defective electrical relay contact

A further potential failure was prevented through the replacement of a faulty hydraulic component

There appears to be no similarities to the issues experienced following the January 2015 commissioning of London

Bridge Low Level (E&F switches design issue) or repeat software/component failures.

In response to the four failures expert engineers have carried out detailed quality assurance checks and

recommended:

Measures to optimise the performance and reliability of points

Enhanced maintenance practices for the clamp lock point design

Additional inspection and monitoring regimes

There is a high level of confidence within the Route and TLP teams that these measures will deliver a solid state of

reliability going forward.

The lessons learned from this investigation will be incorporated into future Thameslink stages.



of 11

1.0) OBJECTIVE

The intention of this report is to:

To review the point end failures, the subsequent inspections and remedial work following the

commissioning and introduction of platforms 7, 8 and 9 at London Bridge station.

Draw conclusions focusing upon the failures, the maintenance activities preceding and following HL09.

Provide recommendations to prevent reoccurrence of similar issues.

2.0) BACKGROUND

On the 29th August 2016, platforms 7, 8 and 9 opened to the public. At the same time the usage of some of the

assets on the Charing Cross lines changed significantly as trains were routed to different platforms. In particular the

usage of 8 point ends have changed significantly; before the Thameslink blockade these points were swung on

average 13 times per week, in the week following the 29th August 2016, these points were swung an average of 293

times per week (see Appendix 1). In preparation for this change, quarterly maintenance services were completed

and swung multiple times to make sure there were no issues. The Remote Condition Monitoring (RCM) was

reviewed to confirm this.

In the two weeks following the 29th August, 2016, there have been four failures on these point ends accruing 25,000

minutes delay. London Bridge Delivery Unit, with the support of Works Delivery, Route Asset Management team

and the Thameslink Project, embarked on a series of inspections to ascertain root causes and take remedial actions

where required. These inspections were followed by further inspections undertaken by the National experts

2.1) Infrastructure Scope and Technical Information

The infrastructure assets considered as part of this report are limited to operational points either side of platforms 7,

8 and 9 at London Bridge Station and have three differing types of Point Operating Equipment (POE). TLP specified

In Bearer Clamp Lock (IBCL) on concrete bearers as the POE of choice, however in situations where the conductor rail

has to proceed past the switch tips, a HW2000 point machine is instead utilised. Two Rail Clamp Point Lock (RCPL)

operated points on wooden timbers are a temporary installation until a later stage, whereupon they will be

removed. A brief history of the 8 point ends can be seen in Table 1 below, a schematic of the layout around

platforms 7, 8 and 9 can be seen in Figure 1.

Table 1: Point Operating Equipment and Significant Dates

Point Number

POE

Installation stage Commissioned Last Service (Quarterly)

TL7151 IBCL 03/10/2015 January 2016 14/07/2016 TL7152 RCPL 17/10/2015 January 2016 14/07/2016 TL7154 RCPL 17/10/2015 January 2016 14/07/2016 TL7177 HW2000 26/09/2015 January 2016 27/07/2016 TL7178 IBCL 12/09/2015 January 2016 27/07/2016 TL7183 IBCL 12/09/2015 January 2016 29/07/2016 TL7187 IBCL 12/09/2015 January 2016 29/07/2016 TL7188 IBCL 12/09/2015 January 2016 29/07/2016

Points TL7181 remain Clipped and Padlocked out of use and detected until full commissioning at a subsequent stage.



of 11

Table 2: Route Supported Remedial Actions and Observations

4.2) Critical Points: Specialist Inspections

On the 9th September, 2016 the two National experts for both Clamp locks and Switches and Crossings (S&C) visited

three point ends, 7151, 7152 and 7154. These points were selected as they are 3 of the 4 most utilised point ends

within this area and being clustered closely together provided the best possible access arrangements in the limited

time available. The team’s objective was to identify any potential faults or underlying conditions which were

previously missed by the maintenance based teams. A summary of the findings can be found in Table 3 below:

Table 3: Summary of Inspections Undertaken by the National Specialists and Findings

Point End Oil Level SimalubeDetection

Adjustment

Lock slide

protrusionTight Lock RAM issue

Debris

Nearby

7151 - 4xchanged

7152 1xchanged

Air appears to

be leaking (no

oil leak)

7154

Pot detached

(not

corrected)

7177Reverse lock

adjusted

7178

Nearly in red -

topped up (no

leaks)

7183Pot detached

and refixed

7187

Nearly in red -

topped up (no

leaks)

7188

Below red, 1lt

added (no

leaks)

Lube can loose

and retighten

Adjusted

detection

following test

013

L/H side

24mm, RHS

23mm

Paper moved

Point EndLock slide

protrusion

Lock

slide/lock

arm gap

TerminalsSchwihag

Rollers Back drive

Stretcher

BarRCM Oil level Hydraulic hoses

7151Adjusted on

site

Lengthen by

3mm. Now

corrected.

RCM not

working. Now

corrected.

7152

Insufficient

protrusion on

LHS currently

24mm should

be >24mm.

Actuator needs

packing

Insufficient

gap on RH

lock body

< 3mm

1) Rear crank

incorrectly

setup.

2) Stabilisation

plate moving

and requires

packing

Low

required

topping up

(600ml

added)

1) Hoses

require

replacement.

2) Split pins

retaining ram

incorrectly

installed.

7154

Insufficient

gap on LH

lock body <

3mm

Two lock

nuts missing

from

terminals

Clamp lock body



of 11

All rectification works were either undertaken on the night or on the 11th September during a Sunday blockade.

Inspections of the remaining 5 critical ends continue through week 24, with any remaining inspections rectification

works planned undertaken on the 18th September and during week 25.

5.0) DISCUSSIONS AND CONCLUSIONS

5.1) The Four Point End Failures

Of the four failures, it is expected to be confirmed that two of the faults were down to component failures, the

hydraulic ram 7152 points and the relay 7183 points (awaiting confirmation from supplier). It is unlikely that these

faults would have been identified during regular inspections for the following reasons

- The hydraulic ram is covered by a shield which is not removed during any cyclical maintenance activity

and thus not fully visible during the inspection. In addition, even though the seal was damaged, there

was not any significant leakage, therefore the fault was not possible to prevent using Remote Condition

Monitoring.

- Cyclical maintenance is not prescribed for the type of relay which failed on 7183 points. The relays are

replaced on a 10 year cycle. The failed relay was manufactured in Feb 2015 well within this timeframe. In

addition the relay failed instantaneously, thus there would not have been any indication from RCM of

the faulty component.

One point to note from the failure of 7183 points is that it took approximately 40 mins from the fault occurring to

the CCT being contacted whom ultimately discovered the fault. The CCT at London Bridge can provide a vital role in

supporting the diagnosis of failures and resultant repair given their proximity to London Bridge Equipment Room

(LBER). They are able to test circuits entering and leaving LBER and have access to diagnostic equipment that the

Flight Engineers do not. In the case of 7183 failure, the CCT was able to test and rectify the fault. Accordingly it is

important they are formally in the communications protocol for ALL failures within their sphere of influence.

Even though it was it was unlikely to have affected these four failures it should also be pointed out that, there is

incomplete phone coverage within LBER. This is likely to hamper rectification of faults in the future.

The two remaining faults on 7178 and 7153 points were caused by the adjustable cam lock nut not being locked

resulting in the tappet being out of specification. Review of this fixing arrangement has concluded that it is unlikely

for them to come undone of their own accord, but left like that during a previous inspection either undertaken as

part of the regular cyclical maintenance or post HL09 blockade inspections. Potentially there are two underlying

reasons for this:

1) The time available to undertake maintenance during mid-week nights has reduced significantly due to the

constraints of accommodating project works in the area. Prior to onset of the Thameslink Project, there was

approximately 3.5hrs working time, the current comparable working time is 1 – 1.5hrs

2) Staff competency levels. London Bridge Signalling Technicians are expected to hold a wide range of

competencies and are expected to be ‘masters of all’, especially in failure circumstances. By way of

example, in just the eight sets of points considered by this report there are three different types of POE -

Clamp Lock, In Bearer Clamp Lock and HW2000 machine. Even though trained and certificated as



of 11

competent, it is very likely that established ‘best practice’ approaches adopted by some are not widely

shared.

The causes of the post HL09 point failures are dissimilar to those experienced following the TLP project stage LL09 in

the Bricklayers Arms area. These failures occurred straight after commissioning and were predominantly related to

back drive set up issues on long switches, E and F type switches. Although there are two E switches in the 8 point

ends around London Bridge station, these have been commissioned since January 2016 and the failures which

followed HL09, were not related to poor back drive setup.

5.2) Pre HL09 Inspections.

A significant amount of tests and checks were undertaken prior to the launch of HL09 no issues were highlighted

before the launch of HL09. Quarterly inspections undertaken in the month prior HL09, focused upon maintaining

safe operation and checking for common failure modes. Swing rehearsals and RCM reviews were prescribed to

highlight deterioration in the switch performance. It would be very unlikely that precautionary checks would have

identified the causes of the failure. The quarterly inspection does not mandate that the tappets and lock nuts are

checked and all failures, with the exception of the hydraulic ram, which was picked up during a post HL09 health

check, were instantaneous and catastrophic and therefore not predicted by RCM.

It is recommended that the contents of the precautionary maintenance should be reviewed to include inspections

which would prevent the issues identified during the failures and follow up inspections. These should be adopted in

preparation for future project phases. Furthermore, it is recommended that prior to a large commissioning phase or

a significant change is usage, the maintenance frequency should be evaluated with review to increase it if required

even as a short to medium term measure.

An additional fault team/resource was provided by London Bridge Delivery Unit, it should be recognised that this

decision helped minimise impact to service particularly in the case of 7183 points failure. As part of this

arrangement, it is recommended that length of cover should be reviewed and potentially extended if there is a spate

of failures which could extend beyond the additional covers expiry point.

5.3) Post Failure Inspections

Following the first two failures, two inspections regimes were put in place to identify other faults on the 8 critical

point ends. The first series of checks, tests and examinations were undertaken by teams from London Bridge

Delivery Unit and South East Works Delivery. They identified various other faults which could have ultimately led to a

failure. The second review of points undertaken by two the National experts for Clamplocks and S&C identified

further faults which in some cases should have been identified as part of cyclical maintenance or the first series of

inspections. A number of conclusions can be drawn from this:

a) The teams do not have sufficient awareness of the importance of critical settings and measurements on the

clamp lock system, particular in respect to those checked as part fault finding.

b) The best practice checks undertaken by the National S&C expert are not regularly employed within the

London Bridge Delivery Unit, particularly during fault finding investigations.

c) Some of the checks prescribed as part of the route based checks were not fully undertaken. This was

identified by the expert led inspections where faults which should have been identified previously were not.

In response to the three concerns above, it is recommended that staff undertake in-depth classes where they can

learn best practices for setting up clamp locks and fault finding. The training should also focus upon the importance



of 11

of having the correct S&C and back drive geometry required for switches and POE to operate safely and reliably.

Particular attention should be given to the adoption of the procedures in Reliability Improvement Alert 069 (RIA

069). Training should contain a sufficient practical element to allow teams to fully explore and understand the

operation of the POE and S&C system; opportunities the teams do not receive in the time constrained environment

of London Bridge. Before embarking on delivering the further classes, it is recommended that a full gap analysis be

undertaken with the support of the RAM team and TLP team where required.

Due to the criticality of clamp locks around the London Bridge Station and its eastern approaches, consideration

should be given to establishing a dedicated highly competent team focused on keeping these point ends reliable.

To support the adoption and continued adherence of the aforementioned best practices, it is also recommended

that the London Bridge S&T Engineers, Section Managers and Supervisors spend more time out on site with their

teams providing coaching and mentoring where required. If increased supervision was in place, it is also likely that

any reduction in the levels of expertise would have been identified and rectified before it became an issue. It is

important to point out that at present, the S&T team have a significant number of key vacancies which will prohibit

the adoption of this recommendation. This team should be supported wherever possible in filling these posts rapidly

with the correct calibre of staff.

The final point to note on the post failure inspections is that a number of points were found with low hydraulic oil

levels. RCM has oil level monitoring function, this should be utilised and its use briefed to the Flight Engineers.

5.4) Recommendation Summary

The following recommendations have been compiled from the previous Discussion/Conclusion section:

a) Bespoke practical upskilling should be arranged for all staff that maintain, install or fault find on S&C fitted

with clamp locks. This should focus upon improving the knowledge of clamplock and S&C best practices, and

back drive installation/setup, but the full contents of this class should be defined as a more formal gap

analysis exercise.

b) To support recommendation a) S&T engineers, section managers and supervisors should provide more on-

site coaching and mentoring.

c) Consideration should be given to setting up a clamp lock specialist team focused on cascading both S&T and

S&C best practices to assets in the London Bridge area.

d) The current post stage response team support should be evaluated before it finishes with the view to

extending it if there still risk of further failures.

e) As part of the pre-commission phase preparations work, an assessment should be made on each asset to

establish if there maintenance frequency should be enhanced.

f) In the advent of an S&T fault, the CCT should be contacted immediately to assist with fault diagnosis.

g) Full phone coverage should be provided to the entirety of the London Bridge Equipment Room

h) The precautionary maintenance undertaken prior to a project commissioning phase should be reviewed to

make sure that appropriate processes are built in to capture the issues seen in the failures and follow up

inspections

i) The current possession arrangements within the London Bridge area need to be reviewed with a view to

maximising the time available for staff to undertake maintenance.

j) The RCM function for monitoring hydraulic oil level in Clamp Locks should be utilised and monitored.

k) Review implementation of RIA069 and its effectiveness



of 11

6.0) APPENDICES

Appendix 1: Comparison of Point Swings for both Before and After HL09

Appendix 2: Fault List for the London Bridge Critical Point Ends (FMS: Jan 2015 – Present Day)

Point Number Failure date Failure mode Failure cause

TL7151 04/02/2016 No normal detection No fault found

TL7152 11/04/2016 No reverse detection

Tamper damaged cable during engineering works

TL7152 01/09/2016 Points secured normal due to defect found on inspection

Defective seal on hydraulic ram

TL7154

08/06/2016 No normal detection

Loose nut on rear back drive rod

05/09/2016 No normal detection

Locking nut loose on adjustable cam causing detection to be out of adjustment

TL7177 No recorded faults

TL7178 31/08/2016 No normal detection Locking nut loose on adjustable

cam causing detection to be out of adjustment

TL7181 No recorded faults

TL7183 07/09/2016 No normal detection Defective contact on relay

TL7187 17/03/2016

RCM alarm, pump pressure high normal to reverse

Back drive out of adjustment and slide chairs contaminated with ballast

TL7188 05/01/2016 Points no reverse Grit contamination in lock body

Items in bold relate to the post-August 2016 period. Previous fault history from January 2016 commissioning

included for reference.

7151 7152 7154 7177 7178 7183 7187 7188 Grand Average

Total (Jan - Jul inc) 462 358 677 447 420 322 219 190 N/A

Weekly Average (Jan - Jul inc) 15.4 11.9 22.6 14.9 14.0 10.7 7.3 6.3 12.9

Week 35 (Post HL09) 227 516 499 No data 268 504 28 10 293.1

Point Number

london bridge point failures investigation report bridge point failures investigation report draft 7...

Documents