Benefits of fatigue risk assessment tools

13.1    Possible benefits of fatigue risk assessment tools and the bio-mathematical fatigue models that underlie them include:

• They can help assess the likely level of fatigue from a current working pattern, to help decide whether further fatigue reduction measures may be reasonably practicable.
• They can help compare the likely level of fatigue which would arise if changes to a working pattern are being considered, for instance during timetable changes or the introduction of a new train service or infrastructure maintenance regime.
• They can help identify particular shifts or sequences within a working pattern where fatigue is likely to be higher, which helps to efficiently target efforts at reducing fatigue risks. For instance, assessment may suggest that the bulk of duties are unlikely to cause a fatigue problem, but that one particular sequence is likely to cause a peak in fatigue, allowing targeting of that sequence for further investigation and risk reduction.
• They can help identify particular features of work patterns, shifts or sequences which are especially likely to contribute to fatigue. This allows alternative fatigue reduction measures to be considered, and the likely effects on fatigue estimated before making any change - duty holders can use some tools to ‘optioneer’, estimating the likely relative merits of for instance shortening shifts or providing extra or longer breaks.
• They can be used in incident investigation as one approach to determine whether fatigue may have been a contributory factor.
• Some fatigue risk assessment tools can be incorporated into resource planning and monitoring software, to help organisations devise fatigue-friendly rosters more easily and quickly from the outset. Some packages can be tailored to automatically monitor deviations from defined company limits or guidelines (e.g. ‘less than 12 hours between duties’ etc), both in ‘planned’ and ‘actual’ working patterns, making it easier to identify likely fatigue hotspots and investigate causes. Such packages are often used to identify staff with the appropriate, in-date competencies. They can also help managers make more informed decisions when considering overtime, extra duties, or shift exchange, by identifying staff whose working pattern over previous days/weeks means they may, on average, be less likely to be fatigued. However, it is vital that the outputs of such tools are not used in isolation for such decisions, as they only provide a general indication of likely fatigue and cannot consider the many individual factors which can make an individual more or less fatigued than a bio-mathematical model may suggest. Some of these limitations are outlined below.

Limitations of fatigue risk assessment tools

13.2    It is imperative to understand that bio-mathematical fatigue models and the fatigue risk assessment tools which use them have significant limitations, so there are several important notes of caution to bear in mind when considering using fatigue risk assessment tools and their outputs:

• Although bio-mathematical fatigue models and tools based on them can provide a useful indication of the level of fatigue which staff are likely to encounter, it is important that staff using them, and interpreting their output, are aware of the particular tool’s assumptions and limitations.
• The models used in fatigue assessment tools do not ‘know’ the level of fatigue staff will encounter when working a particular pattern, they merely make a mathematical prediction.
• When using a fatigue assessment tool, it is important to understand and think carefully about what the output actually means rather than to assume it produces an authoritative ‘satisfactory/unsatisfactory’ decision. Taking the HSE’s Fatigue and Risk Index tool (FRI) (see Spencer and others (2006) for more details) as an example:
  • The Fatigue Index represents the estimated probability, expressed as a percentage, that a person working the pattern concerned will feel very fatigued at some point during the shift. A fatigue index of 10 therefore means that on average, 1 in 10 people working that pattern are likely to feel very tired. Although this is clearly more desirable than a fatigue index of 50 (meaning half the people are likely to feel very tired), it does not mean that a fatigue index of 10 is risk-free. But it does indicate which of the two working patterns is likely to be less tiring.
  • Similarly, the Risk Index gives an estimate of the relative risk of an incident compared to a reference pattern of 12-hour shifts on a typical two-day, two-night, four-off schedule. A risk index of 1.4 therefore means that there is an estimated 40% increase in risk compared to the reference pattern - better than a risk index of 2.0 (double the risk), but not risk-free. The tool helps compare the likely relative merits of working patterns rather than giving any ‘acceptable/unacceptable’ decision.
  • In 2008, the Health & Safety Laboratory (HSL) produced a report (HSL, 2008) evaluating the UK Rail Sector Initial Fatigue & Risk Index Thresholds, which referred to ‘indicative threshold’ values for the HSE FRI fatigue and risk index outputs. These values reflect what was found to be achievable by the great majority of the rail companies surveyed at the time, rather than a definitive, universal interpretation of good fatigue control. Organisations should not assume that just because FRI analysis of their working patterns produces FRI values below the 2008 indicative thresholds that they need do no more – staff may still be suffering from significant fatigue, and it will often be reasonably practicable to improve fatigue controls further.
  • It should be noted that the Fatigue Risk Index was withdrawn from the HSE website in June 2021. The HSE determined that the software platform on which it runs is an older version of Excel that can no longer be supported and maintained on the HSE website. Additionally, the design of the FRI requires improvement to promote better understanding of its outputs, its limitations, and its role in a Fatigue Risk Management System. In its current format, there have been cases of the FRI being misused to justify work patterns that clearly require further action to reduce fatigue-related risk. FRI users who have access to the FRI in its current format can continue to use it provided they have the necessary expertise and understand the outputs and limitations.
• The choice of any threshold should so far as reasonably practicable be validated against the specific activities of the company. The same level of fatigue may produce very different levels of risk depending on the activity conducted. A sensible approach would be for an organisation to develop its own ‘acceptable’ limitations based on a retrospective analysis of statistical correlations between its performance (or data from similar operations which are representative) and/or safety indicators, and the model outputs (RSSB, 2016 Research Report T1083). However, organisations should still treat any ‘thresholds’ with caution. They may be useful as a rough comparator for giving a general indication of how fatiguing a pattern is likely to be but should not be used as a hard and fast threshold with ‘satisfactory’ below and ‘unsatisfactory’ above.
• Fatigue assessment tools cannot model all the factors which affect fatigue. People naturally vary in how much sleep they need, how easily they are able to adapt their sleeping patterns, whether they are more alert earlier or later in the day, and their personal circumstances. Every work situation brings its own unique combination of individual circumstances which can affect fatigue including age, health, personality, family, domestic and social circumstances, personal preferences, and detailed work demands. Models used in fatigue risk assessment tools cannot account for all these variables and therefore cannot perfectly predict fatigue. The best they can do is give a prediction of likely fatigue.
• In particular, many models assume that staff will be able to get sufficient, quality sleep during off-duty periods. They do not consider that staff may not have been able (or in some cases willing) to get the ‘assumed’ amount of quality sleep before presenting themselves for work. RSSB also note that accumulation of fatigue over extended periods (several weeks) are currently not well represented in the models (RSSB, 2016 Research Report T1083). Hence the importance of devising fatigue-friendly working patterns which encourage sufficient good quality sleep, and of personal accountability, education in sleep hygiene, and a ‘just’ culture which encourages openness about fatigue problems.
• Fatigue risk assessment tools mostly predict the potential for fatigue risk, but do not directly assess the risk of performance issues that may contribute to safety events. Several research projects referenced in RSSB’s 2016 Research Report T1083 have demonstrated that the link between fatigue and safety is neither simple nor linear. It may also differ depending on the type of cognitive and/or physical tasks performed, the possibility to co-operate with co-workers and the use of automated systems.

Summary

13.3    Reviews of fatigue models, tools and their uses (CASA 2014; Dawson and others, 2011 and RSSB, 2016 Research Report T1083) emphasise their limitations, and that they are only appropriate as one element in a wider fatigue risk management system. It is essential that additional strategies are used to identify and manage fatigue to complement this approach for example fatigue awareness programmes for schedulers and staff, fatigue reporting systems and consistency with good fatigue management practices.  

13.4    ORR does not compel the use of such tools and does not endorse or advocate the use of any one tool over another. All have their benefits and limitations, and each organisation should decide for itself which tool best suits their requirements.

13.5    Overall, although fatigue assessment tools are a useful aid to making decisions about fatigue, it is important to consider any assumptions and limitations of the specific tool, and to think carefully about the meaning of their output. Such tools are not a substitute for a comprehensive FRMS, rather they are just one useful component. Trusting the outputs of fatigue models in isolation can result in decisions which either promote fatigue or place un-necessary limitations on work.