8.1 The key actions in the Check part of the framework are Measuring performance and Investigating accidents and incidents (refer to HSE publication HSG65 for general guidance).
Measuring performance
8.2 For effective fatigue risk management, organisations should measure their performance, evaluating how their plans have been implemented, whether risks have been controlled and if the organisation’s aims are being achieved. Actual rather than planned working patterns should be monitored and assessed for deviations from good fatigue management practice, potentially using a fatigue assessment tool. Other methods can also be used to measure fatigue, e.g. simply asking staff or using staff reporting systems and staff surveys. Allowing staff to respond anonymously when appropriate may facilitate open and honest feedback. Keeping records can provide useful fatigue data from shift exchange, overtime, sickness absence and travel time. Ideally, organisations should establish metrics or key performance indicators (KPIs) which reflect the degree of fatigue in the organisation and enable organisation to measure performance over time.
Monitor working patterns – actual rather than planned
8.3 To be effective, an FRMS should incorporate procedures for measuring and monitoring the levels of fatigue of actual, not only planned, working patterns.
8.4 Although changes from planned working patterns should be kept to a minimum, it is recognised that changing circumstances, for instance temporary staff absence, over-running engineering work or disruption due to an incident mean that the patterns staff work may sometimes differ from those originally planned. It is important that these actual working patterns are monitored so that likely effects on fatigue can be assessed.
Monitor deviations from good fatigue management practices.
8.5 It can be difficult to detect fatigue in operational settings because, unlike alcohol impairment, there is no ‘blood test’ to measure fatigue. However, the conditions that contribute to fatigue are well known and can be measured. For instance, deviations from good fatigue management practices (e.g. those outlined in Designing your work patterns) are likely to increase the likelihood of fatigue, so assessing actual working patterns against these good practices and highlighting significant deviations will help identify features of the patterns likely to cause increased fatigue. Samples of actual working patterns can be compared against good practices manually, but this can be time consuming - more advanced work scheduling software packages can be programmed in a tailored way to flag up deviations from specified conditions (e.g. ‘Attention – less than 12 hours between duties’), both during the planning of working patterns and also retrospectively. Using such methods can help identify rosters / depots /departments / roles / individuals etc with higher potential fatigue risks, allowing the company to prioritise its fatigue reduction efforts more efficiently.
8.6 It might also be appropriate to monitor any periods of non-safety critical work that could have a bearing on a safety critical worker’s fatigue and ability to undertake the safety critical work.
Consider using a fatigue assessment tool to assess rosters and/or samples of actual hours worked.
8.7 Similarly, fatigue assessment tools can be used retrospectively on actual working patterns to help identify relative fatigue risks. Some staff scheduling software packages incorporate a fatigue assessment tool which can automatically provide an indication of the estimated fatigue which a member of staff is likely to have experienced from their actual working pattern. Alternatively, but requiring more time and effort, companies may use staff timesheet information (provided it is reliable) to identify those staff working the most hours (e.g. the top 5% of staff by hours worked) and transfer this timesheet information into a fatigue assessment tool. These tools should not be considered in isolation but should be complemented by comparisons with good fatigue management practices as outlined in this guidance, and by seeking feedback from staff on how fatiguing they find the working pattern in reality. More information is provided in Appendix C.
8.8 If it is not reasonably practicable to monitor all actual working patterns retrospectively, companies should use their judgement and other likely sources of fatigue information in deciding how to sample working patterns for further fatigue assessment.
Obtain staff feedback
Ask staff
8.9 Simply asking staff which turns / links etc cause problems is a good starting point – staff often know from experience which patterns they find most tiring and can often suggest why. Asking staff occasionally about fatigue during everyday contacts helps monitor whether fatigue is a concern for them, and why. Collecting simple information such as this may quickly and cheaply reveal the factors contributing most to fatigue and help suggest where schedule changes or extra controls may be needed to reduce fatigue risk.
Fatigue reporting systems
8.10 A non-punitive fatigue reporting system encourages staff to report instances when they feel excessively tired, and if necessary, request relief from duties. These reports contain valuable data, especially when coupled with information about the conditions that contributed to fatigue, such as the work schedule and features for the period leading up to the report. However, subjective reports of fatigue can underestimate the true extent of performance impairment, especially when an individual is already suffering from acute or chronic fatigue due to sleep loss or disruption of daily sleep patterns. A non-punitive reporting system is essential to encourage staff to report fatigue as it is often under reported. RSSB has produced guidance in their document ‘Encouraging your staff to report their fatigue concerns’ (detailed in Further Information). Appendix C provides more guidance on fatigue reporting systems, and Appendix B details features of a positive safety culture to encourage open reporting.
Fatigue surveys
8.11 Staff fatigue surveys are a useful supplement to routine monitoring of fatigue using other methods. It may be appropriate to survey staff fatigue by questionnaire or similar if there have been reports or other information suggesting a particular aspect of the work pattern is making staff fatigued. Additionally, in higher risk operations it is good practice to conduct a survey of staff fatigue across the operation periodically to help assess the effectiveness of existing controls, even in the absence of reported fatigue - the absence of fatigue reports does not mean that fatigue is absent. RSSB (2010, Research Report T699) outlines a method used for a very comprehensive fatigue survey, but elements of this approach may be adapted and used in a simpler survey to glean valuable information on staff perceptions of fatigue and its causes. It may be relatively simple to identify any ‘problem’ shifts or work patterns / features by for instance asking all staff to take a few seconds to anonymously complete a simple fatigue rating score (e.g. the 9-point Karolinska Sleepiness Scale (KSS) see Figure 12.1) before / during / at the end of a shift, with simple identification of the depot / route / link etc, for immediate deposit in a box in the cab/depot. In this way, a large amount of useful information can be easily and cheaply collected on perceived fatigue in the whole of the operation, though the approach obviously requires honesty by all parties. Appendix C provides more advice on fatigue reporting. Overall, a proportionate approach is recommended – it makes sense to focus fatigue survey efforts on areas of the operation likely to involve greater risks from fatigue.
Evaluate data from company records
Monitor trends in shift exchange
8.12 Monitoring trends in shift exchange can help reveal potentially problematic rosters or personal preferences. For instance, an individual may regularly seek to swap a day for a night shift due to domestic circumstances or personal preferences – it may sometimes be possible to accommodate these preferences from the outset when designing rosters, rather than coping with the knock-on effects of informal shift swapping later on.
Monitor trends in overtime
8.13 Excessive overtime levels could have a bearing on workers’ fatigue. Monitoring trends in overtime (whether planned or unplanned) can help reveal individuals, departments, depots, grades etc which are potentially vulnerable to fatigue risk since, other things being equal, excessive overtime suggests inadequate staffing. Overtime may be used as a trigger for more in-depth fatigue assessment. For instance, if a member of staff works more than a particular number of hours overtime in a month, a procedure could be triggered to investigate the reasons, and an assessment of likely fatigue risks made by comparing hours worked against good practice, by using a fatigue assessment tool, and by asking staff about possible fatigue. Payroll savings can be made by evolving rosters to reduce the amount of overtime worked.
8.14 If working hours information for monitoring fatigue is derived from payment systems, employers should ensure that the information accurately reflects actual hours worked. There have been instances where overtime payment systems and unrecorded on-call duty has distorted the true picture of actual hours worked.
Examine sickness absence records
8.15 Examining sickness absence records may reveal trends in absence rates between particular turns, depots, work pattern features etc. Increased sickness absence usually requires sickness cover by the remaining staff, increasing demands on them and their likely level of fatigue (i.e. sickness absence causing fatigue). In addition, increased sickness absence trends are sometimes caused by staff finding these particular turns / features more tiring than others, leading to staff calling in sick on these turns. Investigating the reasons behind such variations may therefore help identify any contributory fatigue problems (i.e. fatigue causing sickness absence).
Monitor travel time
8.16 The nature and duration of time spent travelling should be monitored and, so far as is reasonably practicable, controlled when it could have a bearing on a person’s fatigue and ability to undertake safety critical work. Appendix A provides more information on travel time.
Interrogate other data sources
8.17 Other data, for example errors in procedures, near misses, impact of service disruption on workload and other safety-concern reporting systems can all help form a more complete picture of fatigue and its likely causes. If there are concerns about fatigue in particular parts of the operation, it may be reasonable to supplement self-reports of fatigue from fatigue surveys with other methods to monitor sleep and performance in staff. For instance, while it may not be practical to apply these techniques widely, periodic studies of actual sleep using actigraphs (motion-sensing wristbands) and sleep logbooks can be valuable in more objectively measuring the extent of fatigue across different work patterns or groups. Such approaches are becoming more common in other countries and for instance the airline industry, helping to identify causes of fatigue which can then feed into modified work schedules or other aspects of the FRMS.
8.18 Useful background on methods for measuring fatigue can be found in Appendix C.
Monitor Key Performance Indicators (KPIs)
8.19 The continuous improvement process should include a system for evaluating and reporting the overall effectiveness of the FRMS. Organisations should monitor their metrics or key performance indicators (KPIs) which have been set up to reflect the degree of fatigue in the organisation, to help track the effectiveness of the FRMS over time and for instance between roles, sites etc. Sources of data for these KPIs could include any of those suggested in this section (Evaluate data from company records). The organisation should monitor these metrics regularly, looking for trends over time which may suggest the need for change or validate the effectiveness of existing controls. This could be one role for any Fatigue Safety Action Group or similar joint group. ORR produced a KPI Information sheet in 2017 which may be useful to organisations devising or reviewing their own fatigue KPIs. It outlines a suggested approach for deriving fatigue KPIs, suggests some possible KPIs collated from railway and wider sectors, and provides links to further information.
Investigate the causes of accidents, incidents or near misses
8.20 Incident investigation procedures should include arrangements for assessing whether fatigue may have been a contributory factor in accidents, incidents, and near-misses. It is good practice for investigation procedures to provide prompts on fatigue aspects, to specify what information should be collected relevant to fatigue, and how this information should be assessed. Investigations should collect accurate duty start and end times and associated travel times in the days and weeks leading up to an incident. This information should extend back at least to the last time when the individual was completely rested – this may be several weeks, given the recovery time needed to make up any accumulated sleep debt. The patterns worked can then be assessed for deviations from good fatigue management guidelines (e.g. Designing your work patterns), and a fatigue risk assessment tool may be used to assess likely fatigue (refer to Appendix D). These findings should be supplemented by a ‘reality check’, asking the individuals involved, and individuals with experience of similar work patterns, whether they believe fatigue may have played a role in the incident, and the reasons for this belief. Other information needed includes how successful the individuals were in obtaining sleep in the opportunities available, environmental conditions that may have exaggerated or contributed to fatigue, relevant health or medical conditions etc. Appendix C suggests examples of fatigue report form information which can be incorporated into company incident investigation procedures to gather information on whether fatigue may have contributed to an incident. A just culture encourages constructive, honest input to the investigation procedure – see Appendix B on features of a positive safety culture.