awake institute scientific study 3-20-15

Awake Institute

Evaluation of Fatigue Management Systems

A Study of the PRISM System at the Kumba Kolomela Mine

June 4, 2012

Prepared by Anneke Heitmann, PhD, Awake Institute, LLC

1

CONTENTS

1. Background

1.1. Description of the Evaluated Fatigue Management System……………………. 02

1.2. Study Objectives……………………………………………………………………………………… 04

2. Methods

2.1. System Implementation……………………………………………………..…………………… 05

2.2. Study Design and Measures…………………………………………….………………………. 08

2.3. Study Participants……………………………………………………………………………………. 10

2.4. Statistical Analysis.…………………………………………………………………………………… 11

3. Results of Surveys

3.1. General Assessment …….………………………………………………………………………..…. 13

3.1.1. Benefits…………………………….………………………………………………..… 13

3.1.2. Feasibility/Practicability and Acceptability............................... 18

3.1.3. Sensitivity to Increased Fatigue………………………….…………………. 21

3.2. Fatigue Countermeasure Assessment.……………………………………………………… 22

3.2.1. Countermeasure Usage………………………………………………………... 22

3.2.2. Compliance with Countermeasure Recommendations………….. 24

3.2.3. Countermeasure Effectiveness………………………………………..……. 24

3.2.4. Countermeasure Practicability………………………………….………….. 26

4. Results of Alertness/Performance Testing and PRISM

4.1. PRISM Sensitivity to Increased Fatigue ….………………………………………………… 27

4.2. PRISM System Effectiveness …..….………….……………………………………..………… 38

4.2.1. Comparison of Data from Baseline and Post-Implementation 38

4.2.2. Countermeasure Usage and Compliance .……………………………. 40

5. Summary………………………………………………………………………….……………………………………..… 42

Appendix…..…………………………………………………………………………………………………………………….. 45

2

1. BACKGROUND

1.1. Description of the Evaluated Fatigue Management System

Kumba Iron Ore, LTD, is a recognized leader in mining safety among its peer mining

corporations within South Africa. Kolomela mine is a greenfield open pit iron ore operation set

to become one of the largest open mining sites in the world. The management team at

Kolomela has adopted a mission statement which includes becoming one of the safest mines in

the world. To that end, they are building a culture of safety from the ground up including the

utilization of the latest fatigue management technologies available. The Kolomela mine chose

to evaluate the PRISM fatigue management system because it offered the ability to predict

fatigue risk and monitor fatigue levels on an individual employee basis in a real time

environment and then provide job-specific countermeasures to mitigate the fatigue risk for

employees that exceed a predetermined fatigue threshold.

PRISM - Predictive Risk Intelligent Safety Module - links human fatigue risk prediction software

and validated alertness technologies to reduce schedule-specific risk. The system is interfaced

with common Time & Attendance systems to predict fatigue risk in real time to provide

practical, schedule-specific fatigue mitigation recommendations. PRISM can give a graphical risk

output to workers at clock-in and clock-out, and can also provide to workers and supervisors,

via various media, automatic notifications when specific fatigue thresholds are crossed. In

addition, PRISM data can be used to track worker fatigue in operational units over longer time

periods and it provides several data-driven management tools (e.g., statistical reports) to give a

detailed overview of the workforce fatigue status.

Fatigue risk prediction is based on bio-mathematical models rooted in science related to

circadian rhythms and sleep physiology. Alertness prediction models take into account

individual sleep patterns (duration, timing and quality of sleep), using actual sleep and/or

predicted sleep based on schedule-specific sleep opportunities. The well-established Three-

Process Model of sleep and alertness computes alertness taking into consideration homeostatic

factors (build-up of sleepiness during wakefulness and dissipation during sleep), circadian

factors (time-of-day alertness changes based on the phase of the biological clock and its

adjustment to changes in circadian sleep-wake patterns), and sleep inertia (the transitory

impairment of alertness after wake-up depending on duration of prior sleep and other factors).

The bio-mathematical model of PRISM used in this evaluation study is called FIRM, or Fatigue

Index Risk Measurement. FIRM provides a Fatigue Risk Index for each worker in real-time. The

scale of the Fatigue Risk Index is divided into several risk zones, ranging from Low to Severe,

and each risk zone calls for tailored fatigue mitigation actions. The risk zones can be adjusted to

specific job risk profiles. See Figure 1.

In this study, PRISM calculated risk scores directly from actual historic work hour records in real

time at clock in and clock out and predicted the individual fatigue level for upcoming shifts and

3

commutes based on projected scheduled hours. Notifications/warnings were delivered to

workers and supervisors/managers via SMS. Then fatigue countermeasure recommendations

were communicated after accessing a PRISM station. Details about the specific fatigue

countermeasures can be found in the Method section.

Signia Risk

5 Point

Signia

(Refined)

7 Point

Scale Range

Potential %

Exposed and/or

Diminished

Capacity

Strategy

5 Point

Strategy

7 Point

10 9.51 - 10+ 0%

9 8.51 - 9.50 5%

8 7.51 - 8.50 10%

7 6.51 - 7.50 15%

6 5.51 - 6.50 20%

5 4.51 - 5.50 25%

4 3.51 - 4.50 30%

3 2.51 - 3.50 35%

2 1.51 - 2.50 40%

1 .51 - 1.50 45%

0 .50 - (-.50) 50%

-1 -.51 - (-1.50) 55%

-2 -1.51 - (-2.50) 60%

-3 -2.51 - (-3.50) 65%

-4 -3.51 - (-4.50) 70%

-5 -4.51 - (-5.50) 75%

-6 -5.51 - (-6.50) 80%

-7 -6.51 - (-7.50) 85%

-8 -7.51 - (-8.50) 90%

-9 -8.51 - (-9.50) 95%

-10 -9.51 - (-10+) 100%

Protect

(Watchful/Attentive)

Guard

(Shield/Defense)

Protect

Low

Significant

Guarded

Risk Index Measurement Scale

Optimal

Maintain

(Preserve-Conserve)

Nominal

Low

Maintain

Extreme

Guarded

Significant

High

Guard

Proactive

Severe

Vigilant

Copyright 2001-2009

Vigilant

(Cautious/Alert)

Priority 1

Proactive

(Active/Control)

Severe

High

• These index values require immediate

action. Employee will be asked to

take counter measure before

reporting to work station or continuing

work

• At this index value, employee can continue to work but have awareness

of the risk and utilize counter

measures

• At this index value, employee can

safely report to his work station.

Employees need 1 or more days off to

recover to green status

-5 to -10 SEVERE

-2 to -4 HIGH

1 to -1 SIGNIFICANT

4 to 2 GUARDED

10 to 5 LOW

Critical Job Risk Index Legend

Scale is adjusted to job risk profile!

Figure 1: Zoning of the PRISM Fatigue Risk Index

4

1.2. Study Objectives

This study evaluates the PRISM fatigue management system in an around-the-clock operational

setting. Specifically, it investigates the effectiveness of the PRISM system as a fatigue

management tool, its impact on fatigue awareness, operational feasibility and acceptability of

the system, and validity and sensibility of the PRISM Fatigue Risk Index as a measure of

impaired alertness and performance.

.

5

2. METHODS

2.1. System Implementation

The PRISM evaluation study was conducted at the KIO Kolomela mine in South-Africa.

Figure 2: Images from the Kolomela mine

Testing was conducted at work units with around-the-clock operations. The shift schedule

involved blocks of three and four consecutive 12-hour day or night shifts, separated by two- or

three day breaks with a seven-day break every four weeks. See Figure 3.

WEEKS/ WEEKS/

CREWS CREWS

1 - - D D D - - 1

2 N N N N - - - 2

3 - - - - - D D 3

4 D D - - N N N 4

MON TUE WED THU FRI SAT SUN

WEEKS/ WEEKS/

CREWS CREWS

1 - - D D D - - 1

2 N N N N - - - 2

3 - - - - - D D 3

4 D D - - N N N 4

MON TUE WED THU FRI SAT SUN

Figure 3: Work schedule of the participating work units, involving 12-hour day and night shifts.

Shift changes were at 6:30 am and 6:30pm.

6

PRISM generated various fatigue status reports for the employees. See Figure 4 shows a typical

view of the ‘real time’ fatigue status of everyone clocked into the site. The first line managers

use this to track the status and response of their teams.

Figure 4: Sample of fatigue status report (screen shot)

The implemented fatigue countermeasures included primarily:

- energy drinks,

- exercise breaks,

- napping station (designated napping room).

Fatigue countermeasure recommendations also included drinking water and eating high-

protein snacks, dried fruit and meat. A bright light station was also available, but not used in

the study as there was no occurrence of the higher fatigue levels that called for that measure.

Figure 5: Fatigue countermeasures: Napping station, energy drink, bright light.

Countermeasure assignments were communicated via SMS and needed to be confirmed by the

worker on a PRISM unit within one hour. Depending on the actual fatigue risk level (Significant,

High or Severe), there were three different sets of fatigue countermeasures. See Figure 6. The

Significant level resulted in recommendations for an exercise break, drinking water and an

energy drink and eating a high protein snack. At the High level, recommendations also included

a 20-min nap as well as advisement to work with a partner with good fatigue status. At the

7

Severe level, recommended nap duration was increased to 30 min, and bright light use was

advised. The maximum nap duration of 30 min is based on scientific evidence that longer naps

(e.g., 45 or 60 min) induce increased sleepiness at wake-up (sleep inertia), and would also be

operationally less practical.

All participants attended a fatigue countermeasure training session prior to actual testing. The

training informed participant about the use of countermeasures and the notification process.

See Figures 6 and 7.

Figure 6: Sample slides from Fatigue Countermeasure Training – Mitigation strategies by risk level

� Drink between 11:pm –3:am

� Try to drink with food

� Try NOT to drink after 3:am

� 911 drink lasts 5-6 hours

� You can request 911 drink at shift start if tired

� Requires Supervisors permission – only he has a key to nap station

� Timing to use nap station agreed between you and supervisor

� Used for ‘High’ and ‘Severe’ Fatigue Status◦ At ‘High’, set alarm for 25 minutes

◦ At ‘Severe’, set alarm for 35 minutes

Figure 7: Sample slides from Fatigue Countermeasure Training – Countermeasure user instructions

8

2.2. Study Design and Measures

Data collection was conducted during two study phases, before and after PRISM implementation.

See Figure 8. Each study phase included one full cycle of the roster schedule of 28 days, with both

sets of multiple 12-hour day shifts and multiple 12-hour night shift blocks. Workers were asked to

record their activity pattern (hours of sleep, wake, work and commuting to/from work) in an

Activity Diary for about one month during each study phase. During about the same time period,

they also participated in testing before and after each shift. The testing included an alertness test

battery with Visual Analog Scales for subjective ratings of arousal level, mood, motivation,

concentration and physical fatigue; the Karolinska Sleepiness Scale; and a computerized four-choice

reaction time test (Wilkinson Test). In addition, workers completed a daily Shift Performance Log at

the end of each work shift. See Figure 9. Baseline (pre) testing with PRISM off occurred in June/July

2011 and Post-implementation testing with PRISM on in September/October 2011.

daily Activity Diary (records times of sleep/work/commuting)

Work Shift

Alertness Test Battery Alertness Test BatteryShift Performance Log

Surveys:

Fatigue and Health Surveys (during Baseline and Post-Implementation)

PRISM Evaluation Surveys (after study completion)

Pre/Post-Shift Testing:(for all shifts)

twostudy

phases

BaselinePRISM Off

Post-ImplementationPRISM On

1 2

Figure 8: Study design

9

Activity Log(Example)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Sleep*

Awake

Commuting**

Work I: Regular

Work II: Training

Time of Day

ID#: 28 Date: 06/03/11 Weekday: Friday

6

Wilkinson Response Test

Figure 9: Selected data collection tools

The Visual Analog Scales 1 2 3 were 100-mm lines (on which participants marked their ratings) with

the ends of the scales representing the opposite extremes for each parameter: arousal – very

sleepy / very alert; mood – very bad mood / very good mood; motivation – not motivated at all/

very motivated; concentration – unable to concentrate / able to concentrate very well; physical

fatigue – very fatigued / not fatigued at all. The Karolinska Sleepiness Scale (KSS) 4 is a 9-point

scale: 1 = very alert; 9 = very sleepy, great effort to keep awake, fighting sleep. The daily Shift

Performance Log included fifteen brief questions to assess the overall level of alertness and

performance during the shift, including potential nodding-off events and

accidents/incidents/injuries, and countermeasure use.

1 Folstein MF and Luria R (1973): Reliability, validity and clinical application of the Visual Analog Mood Scale.

Psychol Med 3, 479-486.

2 Lee Ka, Hicks G, Nino-Murcia G (1991): Validity and reliability of a scale to assess fatigue. Psychiatry Research 36

(3), 291-298.

3 Wewers ME, Lowe NK ( (1990): A critical review of visual analogue scales in the measurement of clinical

phenomena. Research in Nursing and Health 13, 227-236.

4 Kaida K, Takahashi M, Akerstedt T, Nakata A, Otsuka Y, Haratani T, Fukasawa K (2006): Validation of the

Karolinska sleepiness scale against performance and EEG variables. Clinical Neurophysiology 117(7), 1574-1581

10

In addition to the activity diary and on-shift testing described above, two types of surveys were

administered: a Fatigue and Health Survey and a PRISM Evaluation Survey.

The Fatigue and Health Surveys were used to describe the study population and to assess

comparability of any potential differences in the participant groups used for Pre and Post

testing. Questions included demographic information (age) and information on sleep, alertness,

medical issues, overtime, and fatigue management. A 52-question version of the Fatigue and

Health Survey was administered to workers, and a shorter version (21 questions) was used for a

smaller group of individuals who held supervisory or management positions (workers in

supervisory positions completed both versions). This survey was administered three times: at

the end of the pre-implementation and post-implementation study phases in 2011, as well as in

fall 2010, one year before the post-implementation testing (early baseline). This early baseline

survey was used for assessing the understanding of fatigue levels and fatigue monitoring before

participants had any knowledge of the PRISM system. The complete Fatigue and Health Survey

for workers with the results from Post testing is included in the Appendix.

The PRISM Evaluation Surveys were administered to workers (26 questions) and to supervisors

and managers (14 questions) at the end of the post-implementation study in fall 2011. The

purpose of this survey was to assess the general benefits and operational

feasibility/acceptability of the PRISM system and specific fatigue countermeasures. In addition

to multi-choice response options, many questions of the surveys included space for verbal

comments.

2.3. Study Participants

The study participants were workers of the Maintenance&Engineering group at Kumba’s

Kolomela mine. Table 1 shows the number of people participating in the survey data collection.

Number of

Participants

Health/Fatigue

Survey

Fall 2010

Health/ Fatigue

Survey

Summer 2011

Health/Fatigue

Survey

Fall 2011

PRISM Evaluation

Survey

Fall 2011

Workers 58 16 25* 25*

Supervisors/Managers 7 4 7 7

Table 1: Number of individuals participating in survey data collection. *Two additional workers who completed the surveys were not yet on the PRISM notification system and

were excluded from the analysis.

All the survey participants were male except for three workers who participated only in the

Health and Fatigue Survey during the earlier baseline in fall 2010. The worker surveys were

completed by shift-working mine workers including some with supervisors function, and the

11

supervisors/manager surveys were completed by shift-working supervisors and some section

managers who were dayshift supervisors.

The group of mine workers who were on PRISM and completed the PRISM Evaluation Survey at

the end of the study included 25 people, with 12 people, 9 people and 4 people in the age

brackets 20-29 years, 30-39 years and 40-49 years, respectively. See Appendix for complete

responses for all survey questions.

The survey results for the workers (Result section) are shown in percentage of workers, rather

than number of workers, to standardize result presentation and adjust for occasional

incidences where a worker skipped a question. The survey results for supervisors/managers,

however, are showing actual numbers of respondents as this study group was very small.

A smaller group of workers participated in the experimental testing involving Shift Logs and test

battery measures. Table 2 shows the number of individuals included in Baseline and Post-

Implementation results. The number of shifts for each participant per test phase varied

between individuals, ranging from just a few shifts in some cases to nearly 30 shifts. The data

from participants with only single or few shifts or with generally unreliable data (e.g.,

individuals who never/rarely changing default response settings) were excluded from the

analysis.

The participant numbers in Tables 1 and 2 are for workers who were monitored on PRISM. Shift

Log and test battery data from a few additional study participants, but who were not on PRISM,

were not included in this report in order to assure that the analysis is are based on comparable

data pools.

Shift Logs

Summer 2011

(Baseline)

Test Battery

Summer 2011

(Baseline)

Shift Logs

Fall 2011

(Post)

Test Battery

Fall 2011

(Post)

17 15 23 18

Table 2: Number of individuals participating in survey data collection

2.4. Statistical Analysis

The statistical analysis included the following tests:

- Chi-Square test for comparing frequencies (e.g., survey data),

- t-test or Mann-Whitney Signed Rank test for comparing means of independent measures (for

data passing or failing the normality test, respectively),

12

- paired t-test or Wilcoxon Signed Rank test for comparing means of paired records (repeated

measures) (for data passing or failing the normality test, respectively),

- correlation analysis (Pearson correlation coefficient) for analyzing intra-individual correlations

between PRISM data and the various experimental test data.

The results of the statistical analyses are shown in the graphs by indicating the level of statistical

significance: ‘***’, ‘**’ or ‘*’ for indicating a statistically significant difference at the significance

level p=<0.001, P=<0.01 or p=<0.05, respectively, ‘(*)’ for a trend just below the 0.05 significance

level, or by ‘n.s.’ for not significant differences. The expression “statistically significant” refers to a

result that is unlikely to have occurred by chance, with “p” indicating the probability that the

difference in the data between two groups may be due to random sampling variability (lower p

indicates stronger result).

13

3. RESULTS OF SURVEYS

3.1. General Assessment

3.1.1. Benefits

Most of the workers (76% - 19 people) and all of the supervisors/managers rated the potential

benefits of the PRISM fatigue monitoring system as “very” or “somewhat beneficial” on the

PRISM Evaluation Surveys, with all but one of the supervisors/managers rating it “very

beneficial”. While workers overall favored the system (no worker selected the response option

“not beneficial at all”), the percentage of individuals selecting the highest rating was higher in

the supervisor/manager group than in the worker group (52% of all worker survey respondents,

and 43% when only considering respondents who also had supervisor status). See Figure 10.

Perc

enta

ge

of W

ork

ers

0

20

40

60

80

100

not beneficialat all

slightlybeneficial

verybeneficial

somewhatbeneficial

not sure

How would you rate the potential benefits of PRISM fatigue monitoring system?

0

Nu

mb

er

of S

up

erv

iso

rs/M

an

ag

ers

0

1

2

3

4

5

6

7

not beneficialat all

slightlybeneficial

verybeneficial

somewhatbeneficial

not sure

How would you rate the benefits of PRISM fatigue monitoring system?

0 0 0

Figure 10: Ratings of the potential benefits of the PRISM fatigue monitoring system, assessed by

workers (left) and supervisors/managers (right)

Workers and supervisors/managers made very positive comments about the PRISM benefits.

Many of the verbal survey comments are included throughout the Result section, in blue and

green boxes for workers and supervisors/managers, respectively. The study team was positively

surprised about the many verbal comments received on the surveys, which in itself indicates

that people cared about the project. “Supervisors acknowledge that we are not robots” is

perhaps one of the most illustrative comments indicating the importance of fatigue

management.

Workers’ Quotes: Benefits of PRISM

“It will reduce the rate of fatigued employees.”

“[It]helps you by telling you what to do when you feel sleepy on the job. Making you aware of your status.”

“How to work against low alertness, i.e., stretching, right things to eat and all that.”

“Supervisors acknowledge that we are not robots”

“Makes you aware of your fatigue levels, in other words, you are not in denial about your fatigue status and

alertness status.”

14

One of the most important benefits of the PRISM fatigue monitoring system is increased safety

awareness. All of the supervisors/managers and most of the workers (84% - 21 people) agreed

that the system increases awareness of job safety and performance, with the remaining

workers not being sure except for one person who disagreed. See Figure 11.

Do you think that PRISM fatigue monitoring system

increases your awareness of job safety and performance?

Pe

rcen

tage

of W

ork

ers

0

20

40

60

80

100

not sureyes no

Do you think that PRISM fatigue monitoring systemincreases awareness of job safety and performance?

Num

be

r of

Mana

gers

0

1

2

3

4

5

6

7

not sureyes no

0 0

Figure 11: PRISM-related increase of awareness of job safety and performance, assessed by

workers (left) and supervisors/managers (right)

All of the supervisors/managers said on the PRISM Evaluation Surveys that PRISM gives them

the ability to manage their employees’ fatigue levels at work, and most of the workers (80% -

20 people) said that it gives them the ability to manage their own fatigue level. See Figure 12.

These questions were also asked on the Fatigue and Health Surveys during baseline and post-

implementation. It showed an increase in the percentage of workers who thought that a fatigue

monitoring system would give them the ability to manage their fatigue, from 73% (baseline one

year prior to post) to 88% (post), indicating that becoming familiar with an actual fatigue

monitoring system lead to more people supporting it. See Figure 13, top panel.

Supervisors/managers, on the other hand, were already quite confident about fatigue

monitoring during baseline, with all participants saying in both test phases that a fatigue

monitoring system would give them the ability to manage their employee’s fatigue.

Supervisors’/Managers’ Quotes: Benefits of PRISM

“People know their fatigue status and tend to be more careful.”

“[PRISM] reduces safety risk, it makes me always wanting to monitor the focus of each team member.”

“If [workers] know their status they can be able to manage the way they work.”

Workers’ Quotes: PRISM and Safety Awareness

“It makes a person to be alert all times and stay focused on your job”

“At least I know what to do to keep me awake even though I fail to be alert sometimes”

15

Do you think that PRISM fatigue monitoring system gives you

the ability to manage your fatigue level while on your shift?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

100

not sureyes no

Do you think that PRISM fatigue monitoring system gives youthe ability to manage your employees' fatigue level while on shift?

Num

be

r of M

an

agers

0

1

2

3

4

5

6

7

not sureyes no

00

Figure 12: PRISM-related ability to manage worker fatigue levels, assessed by workers (left) and

supervisors/managers (right)

Do you think that a fatigue monitoring system gives you the abilityto manage your fatigue level while on your shift?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

100

Baseline

Post-Implementation

yes not sureno

p<0.05

Do you think you have understanding of your fatigue levelsand have the ability to manage it properly while on your shift?

Pe

rce

nta

ge

of

Wo

rke

rs

0

20

40

60

80

100

Baseline

Post-Implementation

yes not sureno

0

p<0.01

Do you think that your manager has an understanding of fatigue levels of your joband provides the ability to manage it properly while on your shift?

Perc

en

tage

of

Wo

rke

rs

0

20

40

60

80

100

Baseline

Post-Implementation

yes not sureno

p<0.001

Figure 13: Pre/post comparisons of perceptions of fatigue and fatigue monitoring. Top: Effects of

fatigue monitoring on ability to manage own fatigue, assessed by workers. Bottom: Workers’ (left

panel) and supervisors’/managers’ (right panel) understanding of workers’ fatigue levels and ability

to manage it, assessed by workers. P-values indicate statistical significance level (Chi-Square test).

To evaluate any changes after the implementation of PRISM, the pre/post Fatigue and Health

Surveys asked participants also whether they had an understanding of fatigue levels and the

ability to manage them. The majority of the workers thought they had an understanding of their

16

own fatigue levels during the post-implementation phase (80% - 20 people) which was

significantly increased from baseline one year earlier (62% - 36 people). A smaller portion of the

workers (44% - 25 people) thought during baseline that their manager had an understanding of

the fatigue levels of the job and an ability to manage fatigue. After PRISM implementation this

percentage of workers being confident in their managers’ understanding of fatigue increased

significantly to 76% (19 people). See Figure 13 lower left and right panels.

This was also reflected in the responses of the supervisors/mangers. During baseline, only four of

the seven participants thought they had an understanding of their employees’ fatigue levels and

an ability to manage it, while almost all (n=6) made this statement on the post-implementation

Fatigue and Health Survey. Similar trends were observed when supervisors/managers were asked

if KIO Kolomela had an understanding of the fatigue levels. See Figure 14.

Do you think you have an understanding of your employee fatigue levels

and they have the ability to manage it properly while on your shift?

Num

ber

of S

uperv

iso

rs/M

ana

gers

0

1

2

3

4

5

6

7 Baseline

Post-Implementation

yes not sureno

Do you think KIO Kolomela has an understanding of fatigue levels of your joband provides the ability to manage it properly while on your shift?

Num

ber

of S

uperv

iso

rs/M

anagers

0

1

2

3

4

5

6

7 Baseline

Post-Implementation

yes not sureno

Figure 14: : Pre/post comparisons of the supervisors’/managers’ (left) and KIO Kolomela’s (right)

understanding of fatigue levels, assessed by supervisors/managers.

While workers and managers thought that a fatigue monitoring system would improve

understanding and management of fatigue (see above), workers interestingly also said that they

would feel better about their work environment when knowing that all employees around them

were monitored for alertness/fatigue. Almost all workers (24 people - all but one) said they

would feel much better or somewhat better, with the majority of them (80% - 20 people) saying

they would feel much better. See Figure 15.

Would you feel better about your work environment knowing

that all employees around you had been monitored for alertness/fatigue?

Pe

rcen

tage

of

Wo

rke

rs

0

20

40

60

80

100

would not mattermuch better somewhat better

Figure 15: Alertness monitoring and workers’ perception of their work environment

17

Fatigue monitoring can help improve working conditions. Almost all workers (24 people – all but

one) thought that PRISM would help management understand workers better (76% - “yes”, 20% -

“somewhat”), and all respondents thought it may encourage other actions by employers to

improve alertness (88% - “yes”, 12% “somewhat”). See Figure 16.

Do you think PRISM fatigue monitoring system may encourageother actions by employers to improve on-the job alertness?

Pe

rce

nta

ge

of

Wo

rkers

0

20

40

60

80

100

noyes somewhat

0

Figure 16: PRISM-related potential effects on management and employers, assessed by workers.

Left: management’s understanding of workers and improving working conditions. Right: Other

actions by employers to improve on-the-job alertness.

In the verbal comment section (see selected quotes below), one individual wrote that PRISM

helps acknowledge the role of fatigue in incidences. One worker commented that the system’s

fatigue assessment is based on work hours rather than “actual conditions”. This statement points

out that it is important to educate workers and management about the theoretical basis of

fatigue prediction and how fatigue models can estimate likely sleep based on sleep opportunities

within a given work schedule. In Section 4, we will compare the PRISM fatigue risk scores with

actual alertness data collect during this study.

Workers’ Quotes: Feeling Better About Work Environment With PRISM

“When working night shift you want to work with someone who is alert”

“If you know that all employees are alert, you will work freely knowing that no one will be injured due to fatigue

related matters.”

Workers’ Quotes: PRISM Helping Management Understand Workers

“It [PRISM] acknowledges that fatigue does play a huge role in the occurrence of incidents (HPI's).”

‘Not really because I think it [PRISM] doesn’t monitor [the] actual conditions [of the] worker, rather judges

[based on] worked hours.”

Do you think the PRISM fatigue monitoring system can helpmanagement understand workers better and improve working conditions?

Pe

rce

nta

ge

of

Wo

rke

rs

0

20

40

60

80

100

noyes somewhat

18

3.1.2. Feasibility/Practicability and Acceptability

Workers and supervisors/managers were asked how feasible or practical PRISM fatigue

monitoring is in their work environment. The majority of respondents (80% of the workers and six

of the seven supervisors/managers) said that it is “usually not a problem” or “not a problem at

all”. No one said that it was “most of the time a problem”, and only some participants (two

workers and one supervisor) thought it could be sometimes a problem. See Figure 17.

How practical would you say it is

to use PRISM fatigue monitoring system in your work environment?

Pe

rcenta

ge o

f W

ork

ers

0

20

40

60

80

100

most of the timea problem

sometimesa problem

not a problemat all

usuallynot a problem

not sure

0

How would you rate the feasibility of conducting PRISM fatigue monitoring system in your work environment?

Nu

mbe

r o

f M

ana

ge

rs0

1

2

3

4

5

6

7


sometimesa problem

not a problemat all


not sure

0 0

Figure 17: Feasibility/practicability of the PRISM fatigue monitoring system, assessed by workers

(left) and managers (right).

Specifically, we asked how difficult or easy login and the SMS notification system were. Most of

the workers (84%) said on the survey that it was “not a problem at all“ or “usually not a problem”

(17 people and 4 people, respectively). See Figure 19. The verbal comment section (see selected

quotes below) included positive individual notes like “easy to use because it just requires to log

the card”. Only a few individuals had trouble with system access (e.g., prolonged login time), and

these technical issues can be easily corrected.

How difficult or easy was it for you tolog into the PRISM system during your shift?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

100


sometimesa problem

not a problemat all


Figure 18: Easiness to log into the PRISM system, assessed by workers.

19

The SMS notification system was viewed by many workers as generally effective for letting

workers know their fatigue status during their shift or on their way home. For example, most of

the workers (84%) said “yes” it was effective (76% - 19 people) or “somewhat effective” (8% - 2

people) during the shift. Usefulness of SMS notifications for the way home were rated favorably

by 72% of the workers (18 people). See Figure 18. The verbal comment section of the survey

includes notes such as “every time” and “very useful”. However, a few individuals’ experience was

somewhat compromised due to setup issues and they commented that they were either not

registered on the system or the system had the wrong cell phone number.

Was the PRISM system SMS notification effective

to let you know your fatigue status during your shift?

Pe

rce

nta

ge

of W

ork

ers

0

20

40

60

80

100

noyes somewhat

Was the PRISM system SMS notification usefulin letting you know your fatigue status on your way home?

Pe

rce

nta

ge

of

Wo

rke

rs

0

20

40

60

80

100

noyes somewhat

Figure 19: Effectiveness of PRISM’s SMS notification system for letting workers know their

personal fatigue status during the shift (left) and on the commute home (right), assessed by

workers.

The general acceptance of the PRISM fatigue monitoring system among the workers was good.

Sixty percent of the workers (15 people) rated it favorably and about another 32% (8 people) said

they were willing to try, with only two workers rejecting the system. See Figure 20, left panel.

Interestingly, only one supervisor thought workers had a favorable attitude and all other

supervisors and managers thought workers were more neutral (marking on the survey either

“open to concept” or “neutral”), while none of them thought employees would reject the

concept. See Figure 20, right panel.

How would you rate your general acceptance of PRISM fatigue monitoring system?

Pe

rce

nta

ge

of

Wo

rke

rs

0

20

40

60

80

100

don't need itneutralfavorable willing to try

0

How would you rate employees' acceptance of PRISM fatigue monitoring system?

Nu

mbe

r o

f M

ana

gers

0

1

2

3

4

5

6

7

reject the conceptneutralfavorable open to concept

0

Figure 20: General acceptance of the PRISM fatigue monitoring system by workers (left) and

supervisors/managers (right)

20

However, all seven supervisors/managers rated the acceptance of PRISM by their own group as

either “favorable” (three respondents) or “willing to try” (four respondents). See Figure 21.

How would you rate supervisors'/managers' acceptance

of PRISM fatigue monitoring system?

Num

be

r of M

anagers

0

1

2

3

4

5

6

7

don't need itneutralfavorable willing to try

0 0

Figure 21: Supervisors’/managers’ acceptance of the PRISM fatigue monitoring system.

The verbal comments on the surveys about acceptance of PRISM were very positive, and some

workers simply summarized it with comments like “I like it very much”. One of the supervisors

made the suggestion that the actual monitoring of the PRISM output should be managed by

another department and not the foremen, indicating that it may be appropriate to have a

dedicated person who can fully focus on the issue of fatigue management. See selected quotes

below.

A potentially sensitive issue related to any kind of fatigue monitoring or fitness-for-duty testing is

privacy. A little over fifty percent of the workers said they were not worried about privacy issues

related to the PRISM fatigue monitoring system, while about one third was somewhat concerned,

with the remaining people being indifferent. Supervisors/managers seemed to worry less about

privacy issues, with only one of the seven respondents stating that he would be somewhat

worried. See Figure 22. Acknowledging a potential difference between workers and

supervisors/managers, the analysis for the workers was repeated without the four workers who

also had supervisory status, resulting in the percentage of not worried workers decrease from

52% to 43%.

Supervisors’/Managers’ Quotes: Acceptance/Feasibility

“Feasibility is good for everyone. Acceptance is good by all workers. All of us will benefit from it (change the

alertness and fatigue level of coworkers).”

“I just felt that it should be monitored by a separate section, somebody to make sure everybody uses the system,

other than the foreman…..Fatigue Center should be managed by another section (on its own). That person will

manage it better as it will be all he has to do.”

Workers’ Quotes: Acceptance of PRISM

“I like it”

“I like it very much.”

21

How concerned are you about privacy issuesrelated to the PRISM fatigue monitoring system?

Pe

rce

nta

ge

of W

ork

ers

0

20

40

60

80

100

would not matternot worried somewhat concerned

How concerned are you about privacy issuesrelated to the PRISM fatigue monitoring system?

Num

ber

of M

ana

gers

0

1

2

3

4

5

6

7

would not matternot worried somewhat concerned

0

Figure 22: PRISM-related privacy concerns of workers (left) and supervisors/managers (right)

3.1.3. Sensitivity to Increased Fatigue

While most aspects of the PRISM system were judged very positively by the workers, a more

controversial issue was PRISM’s sensitivity to increased fatigue. About two thirds of the workers

(67% - 17 people) thought it was very or somewhat sensitive (four workers rated it as very

sensitive). Most of the remaining respondents were not sure and only one respondent thought it

was not sensitive. See Figure 23. Workers expressed their uncertainty in the verbal comment

section, describing situations when they thought PRISM and their own fatigue assessment were

not quite in agreement (see selected comments below). The last of the selected comments

seemed to indicate that some doubt about PRISM’s sensitivity to reduced alertness may stem

from an uncertainty about how the alertness prediction works “by just swiping you card”.

A systematic comparison between the PRISM risk scores and the results from workers’ testing

during the study is detailed in Section 4 of this report. In this context, it is important to

acknowledge that alertness models are progressively becoming improved and can also be tailored

to specific populations based on actual data. While this development work is of a technical

nature, the main important prerequisite for the success of fatigue monitoring is a readiness of

workers, supervisors and managers to change culture and a willingness to accept new

technologies. The survey data presented above have shown that participants generally thought

the PRISM fatigue monitoring was beneficial, practical and acceptable, and it helped increase

fatigue awareness and understanding which in itself is an important accomplishment for

mitigating fatigue. This indicates also that it will be beneficial to include in the training sessions

more information about alertness modeling.

Workers’ Quotes: Privacy Issues

“Some employees would not reveal their fatigue status which is dangerous to other employees.”

“Don’t think it’s that invasive.”

22

From your own experience with the PRISM Fatigue Monitor System, how do you rate its sensitivity to reduced alertness or reduced performance levels

(i.e. did the PRISM system indicate low alertness levels when you felt sleepy)?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

100

notsensitive

verysensitive

somewhatsensitive

not sure

Figure 23: Workers’ perceptions of PRISM’s sensitivity to reduced alertness

3.2. Fatigue Countermeasure Assessment

3.2.1. Countermeasure Usage

The use of fatigue countermeasures was recommended for subjects who reached one of the

critical fatigue thresholds. At Clock Out, the percentage of shifts in the Significant, High and

Severe bracket were 43%, 2% and 0%, respectively (based on a total number of 497 shifts worked

by the study participants between September 25 and November 1, 2011).

The main fatigue countermeasures used in this study were energy drinks (Significant and higher),

exercise breaks (Significant and higher) and napping (High and Severe). Energy drinks were used

most frequently, with about two thirds of the workers stating that they had (at least) one on

almost every night shift, and about one third having one during almost every day shift. Exercise

breaks were taken less frequently with about one third of the workers taking one once during

their work shift series on night shifts and once on day shifts (a work shift series is a block of three

or four consecutive shifts). Less than twenty percent of the participants had an exercise break

during every shift (16% - 4 people during every night shift, 12% - 3 people during every day shift).

Naps were mostly taken during night shifts, and most people who had tried napping did it about

once during their work shift series, with only a few people taking naps somewhat more or less

frequently. On night shifts when fatigue is expected to be highest, all participants had an energy

Workers’ Quotes: PRISM’s Sensitivity to Reduced Alertness

“Sometimes when I am alert it says my fatigue level is high.”

“In most cases it said I'm in low whereas I was extremely tired/fatigued.”

“Sometimes I got an alert that I'm significant, but then I felt very high.”

“Sometimes it says I am in a fatigue zone whereas I'm active and all awake.”

“We seem to doubt as to what can this Prism monitor detect or measure your fatigue level by just swiping your

card how possible is it.”

23

drink at least once, however a considerable portion of the group said that they never had an

exercise break (40% - 10 people) or nap (60% -15 people) during night shifts. See Figure 24.

How often do you have an energy drink?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

Night Shifts

Day Shifts

neveronce per monthduring my

work shift series

almost everywork shift

onceduring my

work shift series

0 0

How often do you take an exercise break?

Pe

rce

nta

ge

of W

ork

ers

0

20

40

60

80

Night Shifts

Day Shifts

neveronce per monthduring my

work shift series

almost everywork shift

onceduring my

work shift series

How often do you take scheduled naps?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

100

Night Shifts

Day Shifts

neveralmost everywork shift

once per monthduring my

work shift series

onceduring my

work shift series

0

Figure 24: Frequency of workers’ fatigue countermeasure usage during night shifts and day shifts.

Energy drinks (top), exercise breaks (middle), napping (bottom)

24

3.2.2. Compliance with Countermeasure Recommendations

Compliance with PRISM’s fatigue countermeasure recommendations was reasonable, but not

optimal. Energy drinks had the highest compliance, with about two thirds of the workers who

received a recommendation saying they always or often followed the recommendations.

Compliance with recommendations for an exercise break or a nap was somewhat lower. See

Figure 25. Note, not all workers actually had received recommendations for each specific fatigue

countermeasure, and napping recommendations which were only issued for High or Severe

fatigue levels were received by only about half of the workers.

How often do you follow

PRISM fatigue countermeasure recommendations?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

Napping

Energy Drinks

Exercise Break

rarelysometimesalways often

Figure 25: Workers’ compliance with fatigue countermeasure recommendations. Percentages are based on the total number of workers who received the specific recommendations at least once.

3.2.3. Countermeasure Effectiveness

Workers and supervisors/managers were asked to rate the effectiveness of the fatigue

countermeasures and the majority (about three quarters of the respondents) thought they were

at least somewhat effective. See Figure 26. Energy drinks were rated as very effective by about

one third of the workers who used them. And nearly all of the supervisors and managers (6 out of

7) rated energy drinks and exercise breaks as very effective. Two supervisors/managers thought

that napping did not make any difference and two did not respond to the question on napping

effectiveness. One has to keep in mind that there was only a limited number of napping

recommendations issued and the study cannot provide a representative assessment of this

countermeasure.

25

How effective are the fatigue countermeasuresfor improving your alertness?

Perc

enta

ge o

f W

ork

ers

0

20

40

60

80

Napping

Energy Drinks

Exercise Break

makes no differencevery effective somewhat effective

0

How effective are the fatigue countermeasuresfor improving alertness?

Num

ber

of M

anagers

0

1

2

3

4

5

6

7 Napping

Energy Drinks

Exercise Break

makes no differencevery effective somewhat effective

0 0

Figure 26: Perception of effectiveness of individual fatigue countermeasures, assessed by workers

(left) and supervisors/managers (right). Percentages are based on the total number of workers who received the specific recommendations at least once.

The diverse opinions about the fatigue countermeasures were reflected in the survey comment

section (see selected quotes below). Particularly the fairly poplar energy drinks received several

comments, noting that the drinks are always available, and also indicating that there were

individual differences in the participants’ liking of the drinks. One individual cautioned against

potential overuse of the drinks.

Supervisors’/Managers’ Quotes: Exercise Breaks

“It helps to stretch when you drive a lot.”

Supervisors’/Managers’ Quotes: Energy Drinks

“Have no complaints from my … shift workers.”

“One needs to monitor members not to overuse energy drinks.”

“For some it works, others not.”

“They are always available.”

Workers’ Quotes: Energy Drinks

“This is a very interesting, helpful prism, I love it very much, but it’s the energy [drink] type that I don’t enjoy

because the drink should have a good taste as well.”

“It [PRISM] does work, but the only problem is the energy drinks, they are not making any difference.”

26

3.2.4. Countermeasure Practicability

Supervisors/managers were asked about the practicality of the individual fatigue

countermeasures in their environment. Most seemed to think that energy drinks and exercise

breaks did not pose a problem at all, while most of the respondents thought napping could be

some problem. No respondent said that any of the fatigue countermeasures were a big problem.

See Figure 27.

How practical would you say is it to use the fatigue countermeasure in your environment?

Nu

mbe

r of

Ma

na

ge

rs

0

1

2

3

4

5

6

7 Napping

Energy Drinks

Exercise Break

not sureit's abig problem

not a problemat all

could besome problem

00 0 0 0

Figure 27: Practicality of individual fatigue countermeasures, assessed by supervisors/managers

Napping elicited several verbal comments on the surveys (see selected quotes below), some

relating to nap duration and nap location, and others commenting on policy issues. Interestingly,

one manager (who was not working night shifts himself) was not supporting sleep/napping at

work at all, which underscores the complexities of implementing fatigue management and

related cultural change.

It is indicated that the implementation of the napping countermeasure may require further fine-

tuning (e.g., mitigation of sleep inertia, considerations for other locations) and a longer time to

gain more experience with more workers taking actual naps in order to better assess their

practicality and effectiveness.

Supervisors’/Managers’ Quotes: Napping

“[Policies should include] disciplining action when caught sleeping, controlled sleeping is permitted”

“Sleep required at work should result in an investigation. Bad habits at home must not be tolerated, allowing

people to "rest" at work…..You should sleep at home, work at work”

“People don't want to sleep in nap station, they prefer to have a nap in their ldv [light duty vehicle] in a safe

place.”

“People feel the nap time is too short (20min), should be around 40 min.”

27

4. RESULTS OF ALERTNESS/PERFORMANCE TESTING AND PRISM

4.1. PRISM Sensitivity to Increased Fatigue

One of the goals of this study is to investigate the relationship between the PRISM output and

test measures of alertness/fatigue and specifically PRISM’s sensitivity to impairment. In a first

step, we investigated PRISM’s sensitivity to alertness by comparing day and night shifts. As known

from the shiftwork literature and workers’ everyday experience, alertness tends to be impaired

during night shifts (e.g. see Akerstedt T (1988): Sleepiness as a consequence of shift work. Sleep 11, 11-34). This was

verified for this operation, using the data from alertness testing, and we investigated whether

this expected difference between day and night shifts would hold true in the PRISM data (see

below). The data from all subjects and shifts in a given test condition (day, night, baseline, post-

implementation) were pooled and treated as independent for statistical purposes, unless noted

differently in the text. No day-night differences were found in the Wilkinson four-choice reaction

time test. It was concluded that the variability of the data was too high to be sensitive to

alertness under the specific field study conditions (e.g., possible distractions during test, limited

practice time, etc.), and the Wilkinson data are therefore not included in this report.

The Shift Log analysis included a total of 196 logs for Baselines (104 night shifts, 92 day shifts),

and a total of 389 Shift logs for Post-Implementation (233 night shifts, 156 day shifts). Subjects

rated their overall on-shift alertness and performance on the daily Shift Performance Log which

they completed after every shift during Baseline and Post-Implementation testing. The Shift

Performance Log included a 5-point rating scale for alertness, ranging from ‘very alert’ to ‘very

sleepy’, and the results for this scale are shown in Figure 28. Day shifts and night shifts differed

(Chi Square test) during both test phases (Baseline and Post-Implementation), with a higher

percentage of shifts with ‘very alert’ ratings for day shifts. In both test phases, there were very

few ‘very sleepy’ ratings for day shifts as well as night shifts (1-3% of shifts).

Alertness During Shift (Baseline)

Perc

en

tag

e o

f S

hifts

0

20

40

60

80

Night Shifts

Day Shifts

Alertness Rating

**

veryalert

moderatelyalert

moderatelysleepy

neither alertnor sleepy

verysleepy

Alertness During Shift (Post-Implementation)

Perc

enta

ge

of

Shifts

0

20

40

60

80

Night Shifts

Day Shifts

Alertness Rating

***

veryalert

moderatelyalert

moderatelysleepy


verysleepy

Figure 28: Alertness ratings (Shift Log) during night shifts and day shifts, shown separately for

Baseline (left) and Post-Implementation (right).

28

Participants also rated their ability to focus or concentrate on the daily Shift Performance Logs

using a four-point scale ranging from ‘excellent’ to ‘poor’ (see Figure 29). Similarly, night shifts

and day shifts were different with a higher percentage of shifts with ‘excellent’ ratings for day

shifts and few or no ‘poor ratings’. The daily Shift Performance Logs also indicated differences

between day and night shifts in ratings on mental exhaustion (for Baseline and Post-

implementation) and in ratings on overall performance (Post-Implementation only).

Concentration/Focus During Shift (Baseline)

Perc

en

tag

e o

f S

hifts

0

20

40

60

80

Night Shifts

Day Shifts

Concentration/Focus Rating

**

fairexcellent good poor

0 0

Concentration/Focus During Shift (Post-Implementation)

Perc

en

tag

e o

f S

hifts

0

20

40

60

80

Night Shifts

Day Shifts

Concentration/Focus Rating

***

fairexcellent good poor

Figure 29: Ability to concentrate/focus (Shift Performance Log) during night shifts and day shifts,

shown separately for Baseline (left) and Post-Implementation (right).

Perhaps the most revealing question of the daily Shift Performance Logs was asking participants

to report whether they struggled to remain awake or briefly nodded off during the shift by a

simple yes/no answer (see Figure 30).

Prevalence of Impaired Alertness:

Self-Reported Nodding Off

Perc

enta

ge o

f S

hifts

0

10

20

30

40

50

Night Shifts

Day Shifts

Baseline Post-Implementation

**

**

Figure 30: Prevalence of shifts with self-reported ‘nodding-off or struggling to remain awake’

(Shift Performance Log) during night shifts and day shifts for Baseline and Post-Implementation.

29

Alertness impairment (nodding off/struggling to remain awake) was significantly more frequent

for night shifts (29% in Baseline, 21% in Post-implementation) as compared to day shifts (10% in

Baseline and Post-Implementation).

The differences between day and night were also seen in the results of the test battery data

(Visual Analog Scales for Arousal, Mood, Motivation, Concentration and Physical Fatigue, and

Karolinska Sleepiness Scale) (see Figures 31 and 32). The analysis included a total of 157 shifts for

Baseline (83 night shifts, 74 day shifts) and a total of 305 shifts for Post-Implementation (184

night shifts, 121 day shifts).

Visual Analog Scales (Baseline)

VA

S S

cale

0

20

40

60

80

100

Night Shifts

Day Shifts

*** ***n.s.***

Arousal Mood Concen-tration

Motivation Physic.Fatigue(scale inverted)

Visual Analog Scales (Post-Implementation)

VA

S

Scale

0

20

40

60

80

100

Night Shifts

Day Shifts

*** ***



**

Figure 31: Ratings of arousal, mood, motivation, concentration and physical fatigue (Visual Analog

Scales ranging from 0 to 100; inverted scale for Physical Fatigue) for night shifts and day shifts,

shown separately for Baseline (left) and Post-Implementation (right).

Karolinska Sleepiness Scale

KS

S

Sco

re

1

2

3

4

5

6

7

8

9

Night Shifts

Day Shifts

** ***


veryalert

verysleepy

Figure 32: Sleepiness scores (9-point Karolinska Sleepiness Scale) for night shifts and day shifts

during Baseline and Post-Implementation.

30

Arousal, Mood and Concentration were significantly lower, and Physical Fatigue and the

Karolinska Sleepiness Score were significantly higher on night shifts as compared to day shifts

during both Baseline and Post-Implementation. Statistically lower Motivation was found on Post-

Implementation night shifts as compared to day shifts. The results in Figures 31 and 32 are based

on data sets that, similar to the Shift Log analysis, pooled the data from all subjects and shifts in a

given test condition and treated them as independent for statistical testing.

Distribution of Arousal Data (Post-Implementation)

Visual Analog Scale Bins (Arousal)

0-9.

9

10.1

0.9

20-2

0.9

30-3

0.9

40-4

0.9

50-5

0.9

60-6

0.9

70-7

0.9

80-8

0.9

Perc

enta

ge o

f R

esponses

0

5

10

15

20

25

30

Night Shifts

Day Shifts

verysleepy

veryalert

90-1

00

Figure 33: Distribution of arousal ratings (Visual Analog Scale) for night shifts and day shifts during

Post-Implementation. Results are displayed in percentage of total responses.

Distribution of Karolinska Sleepiness Scale Data (Post-Implementation)

KSS Score

1 2 3 4 5 6 7 8 9

Pe

rce

nta

ge o

f R

esp

on

se

s

0

5

10

15

20

25

30

35

Night Shifts

Day Shifts

veryalert

verysleepy

Figure 34: Distribution of sleepiness ratings (Karolinska Sleepiness Scale) for night shifts and day

shifts during Post-Implementation. Results are displayed in percentage of total responses.

31

Figures 33 and 34 show the distribution of individual ratings for two selected parameters: arousal

(Visual Analog Scale) and sleepiness (Karolinska Sleepiness Scale). The figures illustrate the

differences between day and night shifts seen in Figures 31 and 32, but also demonstrate that the

majority of the responses were in the ‘alert’ half of the scale. The observation guided some of the

subsequent analysis to focus on impairment (e.g., shifts with nodding-off reports) and also limits

somewhat the potential for correlations between test parameters and PRISM data.

Having confirmed the expected differences between day shifts and night shifts in the test

measures, we were investigating whether these differences could be also found in the PRISM

records. Two sets of PRISM records were analyzed, one set corresponding to the shifts with

alertness testing (see Figure 35) and another, bigger data set including all shifts with PRISM

records during the timeframes of Baseline (n=357) and Post-implementation (n= 603), regardless

of presence of test data for these shifts (i.e., also including PRISM data from employees who did

not participate in alertness testing) (see Figure 36). For this bigger PRISM data set of study

participants and non-participants, day and night shifts were defined by the shift end time in the

PRISM records (5-8pm and 5-8am, respectively).

Figures 35 and 36 show the percentage of shifts and absolute shift numbers for each of the five

PRISM zones, ‘severe’ (-10 to -5), ‘high’ (-4 to -2), significant’ (-1 to +1), ‘guarded’ (+2 to +4) and

‘low’ (+5 to +10). For day shifts, PRISM Values were mostly in the ‘significant’ and ‘guarded’

zones. Data for night shifts show a shift towards ‘significant’ and ‘high’. Only two isolated records

were in the ‘severe zone’, (both at ‘-5’) and no records in the ‘low’ zone. The differences between

night shifts and day shifts were statistically significant (Chi-Square test).

PRISM Fatigue Zone

severe high signif. guarded low

Perc

enta

ge o

f S

hifts

0

10

20

30

40

50

60 Night Shifts

Day Shifts

06

47

39

0

PRISM Zone Frequency (Baseline)

2

23

59

21

0

***

includes only shifts withconcurrent alertness testing

PRISM Fatigue Zone


Perc

enta

ge

of S

hifts

0

10

20

30

40

50

60 Night Shifts

Day Shifts

06

85

65

0

PRISM Zone Frequency (Post-Implementation)

1

30

139

63

0

***


Figure 35: Frequency of PRISM zones (severe, high, significant, guarded, low) for night shifts and

day shifts, shown separately for Baseline (left) and Post-Implementation (right). Bars show

percentage of shifts in each PRISM zone, and absolute shift numbers are indicated above bars.

Includes only PRISM records from study participants and shifts with concurrent Shift Performance

Log testing.

32

PRISM Fatigue Zone


Pe

rce

nta

ge

of S

hifts

0

10

20

30

40

50

60 n=183

2

34

106

41

0

Night Shifts (Baseline)

PRISM Fatigue Zone


Perc

en

tage

of S

hifts

0

10

20

30

40

50

60 n=174

0

17

7681

0

Day Shifts (Baseline)

PRISM Fatigue Zone


Pe

rcen

tage o

f S

hifts

0

10

20

30

40

50

60 n=308

0

42

180

86

0

Night Shifts (Post-Implemenation)

PRISM Fatigue Zone


Pe

rce

nta

ge

of

Sh

ifts

0

10

20

30

40

50

60 n=295

07

164

124

0

Day Shifts (Post-Implementation)

Figure 36: Frequency of PRISM zones (severe, high, significant, guarded, low) for night shifts (left)

and day shifts (right). Upper panel: Baseline. Lower panel: Post-Implementation. Bars show

percentage of shifts in each PRISM zone, and absolute shift numbers are indicated above bars.

Includes all PRISM records during the two study phases for both study participants and non-

participants.

The averages of the actual PRISM Values corresponding to Figures 35 and 36 are shown in Figure

37. PRISM Values for night shifts were significantly lower (indicating increased fatigue) as

compared to day shifts (Mann-Whitney test) for both Baseline and Post-Implementation data.

33

PRISM Value (Average)

PR

ISM

Valu

e

-1

0

1

2

3

Night Shifts

Day Shifts

Baseline(n=197)

Post-Implementation(n=389)

*** ***


PR

ISM

Va

lue

-1

0

1

2

3

Night Shifts

Day Shifts

Baseline(n=357)


All Shifts(n=1780)

*** *** ***


Figure 37: Average PRISM values for night shifts and day shifts during Baseline and Post-

Implementation. Lower PRISM values indicate lower alertness. Left panel: Includes only PRISM

records from study participants and shifts with concurrent alertness testing (Shift Performance

Log). Right panel: Includes all PRISM records for both study participants and non-participants

during Baseline, Post-Implementation, as well as for the entire PRISM recording time (Baseline,

Post-Implementation, and transition time).

As noted, the above statistical results for test parameters and PRISM Values are based on the

assumption of independency of the pooled data. To verify that the results were not due to

imbalances in the number of shifts contributed by each participant, the analysis was repeated

based on within-subject means across all shifts in a given test condition (day, night, Baseline,

Post-Implementation) of a given participant. This way each subject contributed only one value

per test parameter and test condition, and paired means from subjects who contributed data to

both conditions for the comparison were analyzed by tests for repeated measures (e.g., paired t-

test). Although this dramatically reduces the sample size for the statistical comparisons (10

subjects; subjects with only one or two shifts per test condition were excluded), and while

eliminating intra-individual variability between shifts, significant day-night differences were still

clearly seen in the PRISM data, Karolinska Sleepiness Scale and in most of the VAS measures.

The aim of ‘fitness-for-duty’ testing is to detect impairment. PRISM values in the ‘high’ and

‘severe’ zones were considered to indicate impairment. Figure 38 shows that impairment was

clearly more frequent during night shifts in both Baseline (over 20%) and Post-Implementation

(over 10%).

34

Prevalence of Impairment:

Percentage of Shifts with High and Severe PRISM Values

0

10

20

30

40

50

Night Shifts

Day Shifts

Baseline(n=197)


Pe

rce

nta

ge

of

Sh

ifts

includes only shifts with concurrent alertness testing

Prevalence of Impairment:

Percentage of Shifts with High and Severe PRISM Values

Pe

rce

nta

ge

of

Sh

ifts

0

10

20

30

40

50

Night Shifts

Day Shifts

Baseline(n=357)


All Shifts(n=1780)

Figure 38: Frequency of impaired PRISM values (PRISM zones ‘High’ and ‘Severe’) for night shifts

and day shifts during Baseline and Post-Implementation. Left panel: Includes only PRISM records

from study participants and shifts with concurrent Shift Performance Log testing. Right panel:

Includes all PRISM records for both study participants and non-participants during Baseline, Post-

Implementation, as well as for the entire PRISM recording time (Baseline, Post-Implementation,

and transition time).

Shifts with low (fatigue-indicating) PRISM Values (PRISM zones ‘severe’ and ‘’high’; n=68) were

associated with more frequent reports of ‘nodding-off/struggling to remain awake’ on the Shift

Logs (32% of the shifts) as compared to shifts with high PRISM Values (PRISM zones ‘low’ and

‘’guarded’; n=188) (13% of the shifts) (Figure 39). For this analysis, the data from all day and night

shifts, from both Baseline and Post-Implementation were combined.


Self-Reported Nodding Off for Low and High PRISM Values

Pe

rcen

tag

e o

f S

hifts

0

10

20

30

40

50

PRISM Value '+2' and Higher (high)

PRISM Value '-2' and Lower (low)

Figure 39: Prevalence of shifts with self-reported ‘nodding-off/struggling to remain awake’ (Shift

Log). Comparison between shifts with medium/high PRISM Values (PRISM zones ‘low’, ‘guarded’

and ‘significant’ – black bar) and low PRISM Values (PRISM zones ‘severe’/‘high’ - red bar). Data

from both shift types (day, night) and both study phases (Baseline, Post-Implementation) were

combined.

35

When using Shift Log data to define impairment (questions on ‘nodding-off/struggling to remain

awake’), it was found that increased fatigue in the Shift Log data corresponded to lower PRISM

Values (see Figure 40, left panel). The analysis also focused on certain ‘high-risk’ subjects who had

more incidences of impaired alertness (e.g., more shifts with reports of ‘nodding-off/struggling to

remain awake’) than their fellow-workers. For example, one subject who had three shifts (out of

28 shifts) with reported ‘nodding-off/struggling to remain awake’ showed lower (impaired) mean

PRISM Values (mean=-1.33) for these shifts as compared to the mean PRISM Values for the shifts

without these reports (mean=+0.6). The example of another subject who reported reduced

alertness on the Shift Performance Log (choosing response options ‘moderately sleepy’ and ‘very

sleepy’) more frequently (5 shifts out of 32) than most co-workers clearly showed a reduced

mean PRISM Value (mean=-2.4, ranging from -5 to -1) for these five shifts (see Figure 40, right

panel).

Average PRISM Value

for Shifts With and Without Reported Nodding-Off

PR

ISM

Va

lue

-1

0

1

2

3

Shifts Without Reported Nodding-Off

Shifts With Reported Nodding-Off


* *

PR

ISM

Valu

e

-3

-2

-1

0

1

2

3

Average PRISM Value By Alertness Level During Shift

(Selected 'High Risk' Subject)

Neither AlertNor Sleepy

(n=11)

Moderately&VerySleepy

(n=5)

Very&ModeratelyAlert(n=18)

Figure 40: Average PRISM values for different alertness levels (Shift Performance Log) during

Baseline and Post- Implementation. Lower PRISM values indicate lower alertness. Left panel:

Comparison of shifts with self-reported ‘nodding-off/struggling to remain awake’ (red bars) and

without self-reported nodding-off. Right panel: Comparison of shifts with different alertness

ratings (Shift Log) for one selected ‘high risk’ subject with increased occurrence of shifts with

reduced alertness.

Similarly to the Shift Performance Log results shown in Figure 39, the test battery data showed

differences between shifts with low and high PRISM Values (Figure 41). Arousal, Motivation and

Concentration were lower and Physical Fatigue and Karolinska Sleepiness score were higher

during shifts with low (fatigue-indicating) PRISM Values (PRISM zones ‘severe’ and ‘high’; n=26)

than during shift with high PRISM Values (PRISM zones ‘low’ and ‘guarded’; n=138). As with the

related analysis of the Shift Performance Logs, the data from all day and night shifts from both

Baseline and Post-Implementation were combined.

36

Visual Analog Scales for High and Low PRISM Values

VA

S

Scale

0

20

40

60

80

100

PRISM Value: '+2' and Higher (High)

PRISM Value: '-2' and Lower (Low)



n.s.* * ** ***

Karolinska Sleepiness Scale for High and Low PRISM Values

KS

S S

core

1

2

3

4

5

6

7

8

9

PRISM Value: '+2' and Higher (High)

PRISM Value: '-2' and Lower (Low)

***


Scales ranging from 0 to 100; inverted scale for Physical Fatigue) (left) and sleepiness ratings (9-

point Karolinska Sleepiness Scale). Comparison between shifts with high PRISM Values (zones

‘low’ and ‘guarded’ – black bars) and low PRISM Values (zones ‘severe’/‘high’ - red bars).

To further investigate the relationship between PRISM Values and test parameters, within-

subject correlations were performed for subjects who provided a reasonably large number of

data pairs of PRISM data and experimental data (Shift Performance Log data for alertness,

concentration/focus, performance, ‘nodding-off/struggling to remain awake’; VAS data for

Arousal, Mood, Motivation, Concentration and Physical Fatigue and Karolinska Sleepiness score).

Correlation between Alertness (Shift Log) and PRISM Value

Alertness Rating

PR

ISM

Va

lue

-3

-2

-1

0

1

2

3

correlation coefficient-0.351

verysleepy

veryalert


example participant #1 (post-implementation data)

moderatelysleepy

moderatelyalert

Correlation between KSS Score (Sleepiness) and PRISM Value

KSS Score

123456789

PR

ISM

Va

lue

-3

-2

-1

0

1

2

3

correlation coefficient-0.641

verysleepy

veryalert



Figure 42: Intra-individual correlations between test measures and PRISM values for one selected

individual (Post-Implementations data;). Left: Alertness ratings (Shift Log); n=29. Right: Sleepiness

scores (9-point Karolinska Sleepiness Scale); n=28. Regression lines are shown in blue.

37

These intra-individual correlations were generally relatively low. The low intra-individual

correlations are not very surprising given the nature of the data sets with only few data points

indicating more extreme fatigue. For illustration, Figures 42 and 43 show the correlations for two

selected subjects who had higher correlations than most other participants.

Correlation between Arousal (VAS) and PRISM Value

Arousal Level

0 20 40 60 80 100

PR

ISM

Valu

e

-3

-2

-1

0

1

2

3

4

correlation coefficient0.449

verysleepy

veryalert


Correlation between Mood (VAS) and PRISM Value

Mood Level

0 20 40 60 80 100P

RIS

M V

alu

e

-3

-2

-1

0

1

2

3

4


verybad

mood

verygoodmood


Correlation between Motivation (VAS) and PRISM Value

Motivation Level

0 20 40 60 80 100

PR

ISM

Va

lue

-3

-2

-1

0

1

2

3

4


notmotivated

at all

verymotivated


Figure 43: Intra-individual correlations between Visual Analog Scale ratings and PRISM values for

one selected individual (Post-Implementations data, n=24). Top left: Arousal. Top right: Mood.

Bottom: Motivation. Regression lines are shown in blue.

38

Overall, the results indicate that PRISM Values and alertness test data show comparable trends

on a group level and demonstrate PRISM’s relevance for detecting impairment. On an individual

level, correlations between PRISM and experimental alertness data were relatively low.

Refinements of the PRISM fatigue management system (e.g., inclusion of prior day’s sleep as

input parameter for risk calculation and added option for cognitive impairment testing) are

aiming to improve PRISM’s sensitivity to impaired alertness, in particular on the individual level.

For example, a new PRISM module was recently incorporated into the PRISM system which takes

into account workers’ prior sleep (entered into the PRISM interface at log-in) in addition to work

hours when computing PRISM values.

4.2. PRISM System Effectiveness

4.2.1. Comparison of Data from Baseline and Post-Implementation

The results from the worker and supervisor surveys, reported in Chapter 3, illustrated several

benefits of PRISM, such as increased awareness of job safety and performance and increased

ability to manage fatigue levels at work. We also compared the data from Baseline and Post-

Implementation testing (Shift Logs, Visual Analog Scales, Karolinska Sleepiness Scale).

Arousal, Mood, Concentration and Physical Fatigue (from the Visual Analog Scales) showed

statistically significant improvements on Post-Implementation night shifts as compared to

Baseline night shifts (Figure 44, left panel). On day shifts (Figure 44, right panel), Mood improved

and Motivation showed a trend towards improvement (significance level just under 0.05,

indicated by ‘(*)’). Sleepiness rated on the Karolinska Sleepiness Scale improved significantly for

both night shifts and day shifts (Figure 45). It should be noted that the improvements were seen

even though participants generally rated their baseline alertness levels - even on night shifts - in

the mid-range of the scales or slightly better. As with the day-night comparisons, the comparison

of Baseline and Post-Implementation was also run using paired comparisons of the subjects who

participated in both test phases (based on within-subject averages across all shifts of a given

subject and test condition). These paired tests, however, did not result in significant differences

between the two test phases, and the relatively low sample size did not grant sufficient statistical

power.

A non-significant trend towards improvement during Post-Implementation night shifts was also

seen in the Shift Performance Log reports of ‘nodding-off/struggling to remain awake’ for night

shifts (Figure 46), with the percentage of night shifts with self-reported ‘nodding-off/struggling to

remain awake’ decreasing from 28% to 21%. Reports of errors and mistakes on the Shift

Performance Log for day shifts also decreased, although this should be interpreted cautiously as

errors/mistakes were disproportionally high during Baseline day shifts. Means of the other Shift

Performance Log measures did not demonstrate differences between Baseline and Post-

Implementation, due to relatively high data variability, fewer response choice options than on the

39

test battery scales (e.g., 5-point alertness scales on Shift Log vs. 9-point scale on Karolinska

Sleepiness Scale) and relatively small differences in the group averages (differences between

Baseline and Post-Implementation were smaller than the differences between night and day

shifts).

PRISM Values were not expected to be significantly different in Baseline and Post-

Implementation (see Figure 47) as the PRISM algorithms used for the study were based on

schedule characteristics only (algorithms did not take into account sleep times or potential

alertness improvements resulting from fatigue countermeasure use).

Visual Analog Scales (Night Shifts)

VA

S

Scale

0

20

40

60

80

100

Baseline

Post-Implementation

** ***n.s.*



Visual Analog Scales (Day Shifts)

VA

S S

cale

0

20

40

60

80

100

Baseline

Post-Implementation



n.s.n.s.

*n.s. (*)


Scales ranging from 0 to 100; inverted scale for Physical Fatigue) for Baseline and Post-

Implementation, shown separately for night shifts (left) and day shifts (right).

Karolinska Sleepiness Scale

KS

S

Score

1

2

3

4

5

6

7

8

9

Baseline

Post-Implementation

*** ***

Night Shift Day Shift

veryalert

verysleepy

Figure 45: : Sleepiness scores (9-point Karolinska Sleepiness Scale for Baseline and Post-

Implementation, shown separately for night shifts (left) and day shifts (right).

40


Self-Reported Nodding Off

Pe

rce

nta

ge

of S

hifts

0

10

20

30

40

50

Baseline

Post-Implementation

Night Shifts Day Shifts

n.s.

n.s.

Figure 46: Frequency of shifts with impaired alertness (self-reported ‘nodding-off/struggling to

remain awake’ - Shift Performance Log) during Baseline and Post-Implementation, for night shifts

and day shifts.


PR

ISM

Va

lue

-1

0

1

2

3

Baseline

Post-Implementation

Night Shifts Day Shifts

n.s.

n.s.

Figure 47: Average PRISM values during Baseline and Post-Implementation, for night shifts and

day shifts. Includes all PRISM records for both study participants and non-participants during

Baseline and Post-Implementation.

4.2.2. Countermeasure Usage and Compliance

Countermeasure compliance was assessed by worker surveys (see Chapter 3) and on Shift

Performance Logs. Energy drinks was the most frequently used fatigue countermeasure and

surveys showed that this fatigue countermeasure had the highest compliance. The Shift

performance Logs showed that energy drinks were used during about 30% of the night shifts

and close to 10% of the day shifts (see Figure 48, left panel). In addition to energy drinks,

workers consumed other caffeinated beverages that were not related to the PRISM

countermeasure (see Figure 48, right panel). Similarly, this occurred more during night shifts

41

(100% of Post-Implementation night shifts) than during day shifts (about 50% of Post-

Implementation day shifts).

Interestingly, the consumption of energy drinks was not always related to actual

recommendations by the PRISM system (see Table 3). In many cases, workers used energy

drinks even though PRISM did not indicate low enough values to trigger countermeasure

assignment, and in other cases energy drinks were assigned, but not consumed, partly due to

temporary unavailability of the drinks. As other fatigue countermeasures (napping, light

station) were rarely assigned and used, the Shift Performance Logs did not provide enough

information on compliance for these fatigue countermeasures.

Overall, the data on frequency of countermeasure assignment/use suggest that the potential

for improvements was somewhat limited, despite which we were able to detect significant

positive effects of The PRISM fatigue management system during Post-Implementation.

Energy Drink Consumption

Pe

rce

nta

ge

of

Shifts

0

10

20

30

40

50

Night Shifts

Day Shifts

Caffeine Consumption (Post-Implemenation)

Perc

enta

ge

of

Shifts

0

20

40

60

80

Night Shifts

Day Shifts

0 21 3 4 5+

Number of Caffeinated Beverages

0

Figure 48: Consumption of energy drinks (left panel) and other caffeinated beverages on night

shifts and day shifts during Post-Implementation.

Energy Drink Consumption (Number of Night Shifts)

Yes No Energy Drink Consumption (Number of Day Shifts)

Yes No

Assigned by PRISM

59

73

Assigned by PRISM

11

65

Not assigned by PRISM

12

89

Not assigned by PRISM

2

78

Table 3: Energy drink consumption and PRISM countermeasure assignment

42

5. SUMMARY

PRISM - Predictive Risk Intelligent Safety Module - links human fatigue risk prediction software

and validated alertness technologies to reduce schedule-specific risk. The system is interfaced

with common Time & Attendance systems to predict fatigue risk in real time to provide

practical, schedule-specific fatigue mitigation recommendations.

This study evaluates the PRISM fatigue management system in an around-the-clock operation

at the KIO Kolomela mine in South Africa. PRISM monitored workers’ fatigue levels and sent

automatic SMS notifications to workers and supervisors when PRISM’s Fatigue Risk Index

exceeded certain thresholds. The notifications included one of three sets of fatigue

countermeasure recommendations, depending on the fatigue severity level. The primary

fatigue countermeasures were energy drinks, exercise breaks and napping. All participants

attended a fatigue countermeasure training session.

Data was collected from participating workers for one cycle of the roster schedule of 28 days,

working both sets of multiple 12 hour day shifts and multiple 12 hour night shift blocks, before

and after PRISM implementation (summer and fall 2011, respectively). Two types of surveys

were administered. Fatigue and Health Surveys were administered to workers and also to

supervisors and managers at the end of Baseline and Post-Implementation study phases, as

well as during an early baseline one year before the Post-Implementation study phase. At the

end of the study, PRISM Evaluation Surveys were completed by the workers and

supervisors/managers. On the test days, data were collected on workers’ activity patterns

(hours of sleep, wake, work and commuting to/from work), volunteers completed an alertness

test battery before and after each shift (various subjective alertness scales and a reaction time

test) and a daily Shift Performance Log at the end of each work shift.

Survey results: The survey results demonstrate many positive aspects of the PRISM fatigue

monitoring systems. General benefits, acceptability and operational feasibility/practicability

were rated favorably by the majority of the workers and supervisors/managers. For example:

All of the supervisors/managers and most of the workers (84%) agreed that the system

increases awareness of job safety and performance (with most of the remaining workers

not being sure).

Similarly, all of the supervisors/managers and most of the workers said that PRISM gives

them the ability to manage their employees’ (or their own) fatigue levels at work,

Workers thought that PRISM would help management understand workers better,

improve working conditions and potentially encourage other actions by employers.

Statistical comparisons between the early baseline and post-implementation data

revealed that the percentage of workers being confident in their own and in their

43

managers’ understanding of fatigue levels increased significantly after PRISM

implementation.

Workers also said that they would feel better about their work environment when

knowing that all employees around them were monitored for alertness/fatigue.

While the general acceptance of the PRISM fatigue monitoring system by workers and

supervisors/managers was good, about one third of the workers were somewhat concerned

about privacy.

About two thirds of the workers thought the PRISM system was very or somewhat sensitive to

reduced alertness (with most of the remaining respondents not being sure).

Results of on-shift alertness/performance testing and PRISM: The data from on-shift testing

(Visual Analog Scales on Arousal, Mood, Motivation, Physical Fatigue; Karolinska Sleepiness Scale;

Shift Performance Log) was used to investigate PRISM’s sensitivity to decreased alertness and

impairment and its effectiveness for fatigue mitigation. Overall, the results indicate that PRISM

output and test data show comparable trends on a group level and the data demonstrate PRISM’s

relevance for detecting impairment. On an individual level, correlations between PRISM output

and test data were relatively low. Refinements of the PRISM fatigue management system (e.g.,

inclusion of prior day’s sleep as input parameter for risk calculation and added option for

cognitive impairment testing) are aiming to improve PRISM’s sensitivity to impaired alertness, in

particular on the individual level. Test data demonstrate improved alertness after PRISM

implementation. Some of the results from the statistical analysis include:

Average alertness levels were significantly lower during night shifts as compared to day

shifts (as seen in the data from the Visual Analog Scales, Karolinska sleepiness Scale and

Shift Performance Log), and this day-night difference was also reflected in the PRSIM data,

with significantly lower PRISM values on night shifts.

Focusing on impairment, it was found that low PRISM values (PRISM zones ‘severe’ and

‘high’) were associated with a higher frequency of self-reported ‘nodding off / struggling

to remain awake’ (Shift Log data) and with significant changes in Visual Analog Scale data

(reduced Arousal, Motivation, Concentration and increased Physical Fatigue) and

significantly increased score on the Karolinska Sleepiness Scale as compared to higher

PRISM values. Similarly, shifts on which participants reported ‘nodding off / struggling to

remain awake’ had significantly lower PRISM values than shifts without such reports.

When comparing Post-Implementation and Baseline, most of the test parameters for

night shifts (Arousal, Mood, Motivation, Concentration, Physical Fatigue and Karolinska

Sleepiness score) and some test parameters for day shifts showed significant

improvements after the implementation of the PRISM system.

44

Countermeasure Use: Energy drinks were used most frequently, with about two thirds of the

workers stating that they had (at least) one energy drink on almost every night shift, and about

one third having one during almost every day shift (survey results). The drinks were used during

about 30% of the night shifts and close to 10% during day shifts (Shift Log results). Compliance

with PRISM’s fatigue countermeasure recommendations was reasonable but not optimal, and

energy drinks had the highest compliance. The majority of the survey respondents thought that

the fatigue countermeasures were at least somewhat effective, and energy drinks were rated

as very effective by about one third of the workers who used them. Nearly all of the supervisors

and managers rated energy drinks and exercise breaks as very effective.

The assessment of the napping and the bright light countermeasures are somewhat limited

because of the relatively low number of issued recommendations for napping and bright light

station. A longer implementation time to gain specific experience with more workers taking

naps, and further fine-tuning of the specific napping recommendations (e.g., napping duration,

mitigation of sleep inertia, considerations for other locations) are recommended to better

assess practicality and effectiveness of napping. The data on frequency of countermeasure

assignment versus actual countermeasure use suggest that there is a potential for improving

compliance. However, the evaluation results showed significant positive effects of the PRISM

fatigue management during Post-Implementation.

The success of fatigue monitoring depends on the readiness of workers, supervisors and

managers to change safety culture and a willingness to accept this concept and new

technologies. Participants in this implementation trial thought a fatigue monitoring system

would clearly help increase fatigue awareness and understanding, and specifically, the PRISM

fatigue management system was beneficial, practical and acceptable in managing fatigue in

this work environment. Test data illustrated that PRISM does correlate with alertness on a

group level, and data showed a clear trend in reducing overall fatigue in participating

workers after PRISM implementation. Refinements of the PRISM fatigue management system

(e.g., inclusion of prior day’s sleep as input parameter for risk calculation and added option

for cognitive impairment testing) are expected to improve its sensitivity on an individual level.

Improvements in execution to achieve higher countermeasure compliance should further

enhance its positive effects for fatigue mitigation.

45

APPENDIX

Worker Fatigue and Health Survey

Post-Implementation Results

(Response counts for all survey questions)

46

SECTION I: CONFIDENTIAL GENERAL INFORMATION

1. How old are you?

Under 20 years old 20 – 29 years old 30 – 39 years 40 – 49 years 50 years or older

0 12 9 4 0

2. What is your gender?

Female Male

0 25

SECTION II: FATIGUE AND HEALTH ISSUES

3. Do you do exercises regularly (i.e., such as brisk walking, jogging, biking, play soccer)?

Do not have a regular exercise

schedule

Exercise, but not on a schedule

(once per week or less)

Exercise regularly

(2-3 times per week)

10 11 4

4. I am told I snore loudly or I awake suddenly gasping for breath while I am sleeping

Never Rarely Sometimes Usually Always

10 2 9 3 1

5. Are you currently under doctor’s care for any medical problem or on any

medication program under doctor’s care?

Yes No

2 23

How often did you experience these problems in the past 6 months:

Almost Never

Quite Seldom

Quite Often

Almost Always

6. Heartburn

7. Digestion problems

8. Irregular heartbeat

13

16

19

5

7

6

7

2

0

0

0

0

47

9. Do you think that eating the proper foods at correct times can help you feel better during

shiftwork?

I don’t know how proper foods and

timing can impact me during

shiftwork

I know about proper foods and timing

but I don’t apply during for my

shiftwork

I know about proper foods and

timing and I apply them during for

my shiftwork

5 16 4

10. Do you think that understanding your fatigue level can help you improve your overall

health?

Yes No Uncertain

22 1 2

11. How many hours of overtime do you typically work in a given week?

How much you have experienced any of the following problems:

Chronic

Problem

Frequently a

Problem

Sometimes a

Problem

Rarely a

Problem

Never a

Problem

12. High cholesterol

13. Diabetes

14. Headaches

15. Trouble Sleeping

16. Eye Soreness

17. Fatigue

18. Stiffness, aches or pain

3

3

3

2

2

3

2

1

0

2

1

5

3

4

2

1

13

7

4

18

11

0

2

4

5

4

0

5

19

19

3

10

10

1

3

0 hours 1-4 hours

5-8 hours

9-10 hours

More than 10 hours

17 5 1 1 0

48

SECTION III: SLEEP, ALERTNESS, AND SAFETY ISSUES

Which sleep length best describes your current situation (after implementation of the PRISM

system):

Does not

Apply

5 Hours 6 Hours

7 Hours

8 Hours or

More

19. How many hours of sleep per

day (usually) do you feel you need

to be alert and well rested?


day are you actually getting,

usually, when you work the day

shift?



usually, when you work the night

shift?



usually, during your days off?

2

4

6

2

2

8

11

2

4

4

5

2

8

6

2

7

9

3

0

12

How would you rate the quality of sleep that you are getting:

Excellent Good

Average

Below Average Poor

23. on holiday or

days off?

24.during nighttime

when working the

day shift?

25. during daytime

when working the

night shift?

10

2

0

11

12

7

3

7

8

0

2

6

0

1

4

26. When you are on holiday or on the long weekend break, do you naturally rise early and feel

best (i.e. alert, energetic) in the morning, or do you like to sleep in late?

Rise early

(i.e., 5-7 am)

Rise late

(9-11 am)

Rise somewhere in-between (after 7

am, but before 9 am)

9 9 7

49

How often did each of the following events occur?:

Several

Times per

Shift

Several

Times per

Week

Several

Times per

Month

Several

Times per

Year

Seldom, if

Ever

27. How often do you find

yourself fighting sleep or briefly

nodding-off while working?



nodding off during breaks?



nodding-off while driving to and

from work?

30. How often do you make

mistakes or errors due to not

paying attention while working

on your current shift?

31. How often do you experience

muscle pain or discomfort while working on your current

schedule?

32. How often do you feel

fatigued, drowsy or sluggish while working on your current

schedule?

33. How often do you feel your

alertness is too low where you

are not effective while working

shifts?

3

2

4

0

1

2

2

9

8

6

2

3

4

3

8

7

5

5

5

9

16

1

4

1

6

7

7

6

4

4

9

11

9

3

1

34. During your last shift cycle, what was the longest number of consecutive hours you went

without sleep?

1-17 18-20 21-23 24-26 27 or more

14 7 1 2 1

35. How often do you take naps during your waking hours off-the-job?

Every Day Several Times per

Week

Once per Week

Seldom take naps

off-the-job

Never take Naps

5 6 4 7 3

50

36. How often do you take unscheduled naps or fall asleep while on the job?

(unscheduled naps do not include any naps recommended by the PRISM system)

Every Day Several Times per

Week

Once per Week

Seldom take naps

off-the-job

Never take Naps

1 3 3 8 9

37. How well adjusted are you to your current shift schedule?

Poorly adjusted

(having lots of problems)

Getting by Slightly adjusted

Well adjusted Very well adjusted

(having no problems)

1 1 6 11 6

38. How do you typically prepare for the first night shift?

Stay up the night

before and sleep

most of the day

Rise at normal time

that day, stay up all

day, then come to work

Stay up all day, but

take a nap prior to

coming to work

Stay up late and

sleep in late for the

previous two days

Other, please

explain in the

comments section

3 6 14 2 0

39. Do you feel that some type of training on how to better adjust to a shiftwork lifestyle

would make it easier to cope with the special challenges of shiftwork schedules?

Yes No

21 4

How many times has each of the following situations occurred:

0 1-2 3-4

5-6

7 or more

40. Have you had any car accidents or

near accidents in the past 3 months?

41. How many accidents or injuries

have you had on the job in the past 3

months?

42. How many near accidents or

injuries have you had on the job in the

past 3 months?

43. Altogether, how many lost days

have you had in the past 6 months as

a result of any accidents or injuries?

20

21

20

24

4

4

4

0

0

0

0

0

1

0

1

1

0

0

0

0

51

44. Where any of these accidents, injuries, or near misses due to fatigue or lack of

alertness?

Yes No Not Sure Does not Apply

3 7 2 13

SECTION IV: FATIGUE MANAGEMENT STATUS

How do you feel about your job:

Very High High Moderate

Low

Very Low

45. How mentally demanding is your job?

46. How physically demanding is your job?

47. How monotonous or boring is your job?

48. How fatiguing is your job?

9

11

1

3

12

6

0

11

3

6

4

4

1

2

7

6

0

0

11

1

49. Do you think you have understanding of your fatigue levels and have the ability to

manage it properly while on your shift?

Yes No Not Sure

20 0 5

50. Do you think that your manager has an understanding of fatigue levels of your job and

provides the ability to manage it properly while on your shift?

Yes No Not Sure

19 1 5

51. Do you think that a fatigue monitoring system increases awareness of job safety and

performance?

Yes No Not Sure

20 0 5

52. Do you think that a fatigue monitoring system gives you the ability to manage your

fatigue level while on your shift?

Yes No Not Sure

22 0 3

awake institute scientific study 3-20-15

Documents

worker fatigue

specific fatigue thresholds

monitor fatigue levels

workforce fatigue status

prism system effectiveness

prism data

prism sensitivity

schedulespecific risk