Any Human Reliability Assessment (HRA) carried out without an adequate Human Error Quantification and Identification (HECI) process will probably be inaccurate. For accurate complex systems risk assessment, it is critically important that the assessor understands the multifaceted nature of human errors in complex systems and is thus aware of all the different types of human – error contributions which can ultimately make an impact upon a system’s risk level. It is also necessary that the assessor be suitably well – armed with a battery of tools with which to identify such error contributions both reliably and comprehensively. This requires a consideration of some of the most adequate taxonomies or classifications of human error. A new generalised HECI framework will then be described in this paper which deals with many different forms of human error at different levels of analysis. This framework and its tools rest heavily upon already existing techniques and ideas, never attempting to put these into a more comprehensive and coherent framework. Key words: human error, human reliability, performance shaping factors.
1. INTRODUCTION
The following sketches the history of HRA in terms of the predominant techniques and developments over the past three decades. The historical perspective is not necessary for the practitioner, but is included out of interest as it shows the trends that have driven human reliability over the past three decades.
The origin of human-reliability analysis is usually traced back to the American Institute for Research’s database of human error probabilities (Munger et al, 1962). This involved a set of Human Error Probability (HEP) that covered both operable equipment and system design variables. It was Swain, in 1967, who later showed this approach to be inadequate mathematically. The first formal attempt to include task/environmental variables, along with human-engineering-design characteristics, for estimating HEPs was made by Swain, et al 1983, with the Sandia human error rate bank (SHERB) (see Rigby, 1976), which used Performance Shaping Factors (PSFs) to guide the analyst in deriving the human-error rate for a particular task. In the late 60s and early 70s, furthermore, human-error data concerning input errors in naval operations was collected (the operational recording and data system (OPREDS), Coburn, 1971). Other significant attempts at data-bank-driven approaches included the Bunker-Ramo tables (Mills and Hatfield, 1974) and the technique for estimating personnel performance standards (TEPPS, Blanchard et al, 1966).
Thus, during the first decade of HRA, there was a desire to create human-error databanks, by way of a parallel to those being successfully created for hardware components.
There was also a realisation of the need to consider PSFs. By the end of the first decade of HRA, however, it was being realised that the data-bank approach was not working. In 1975, the WASH-1400 Reactor Safety Study (Rasmussen, 1975) was compiled (the prototype of the Probabilistic Risk Assessment) which included a still used human error database. At this time, practical examples of the Technique for Human Error rate Prediction (THERP) were beginning to emerge (Swain, 1976). In parallel, in the early 70s, Siegel et al (1974) developed digital simulation methods for estimating the reliability of man-machine systems: these included a consideration of human errors created as a result of inadequate performance.
Askren & Regulinski (1969) considered human error to be dependent on time, and they based their approach on the use of the traditional reliability ratio of mean time to failure. In the 70s, therefore, three strands of HRA research were evolving. The first involved a continuation of the database approach in the form of the THERP system, which, although it is a highly decompositional approach by modern standards, was far more integrative than the earlier data-bank approaches, dealing far more with tasks than with elemental behavioural units. The second strand was the simulation approach, which offered stochastically to simulate the human operator’s reliability by using distributions of performance times combined with a Monte Carlo style simulation. The third strand was the time-reliability approach, again a parallel development to approach the reliability assessment, involving the modelling of the ”mean-time-to-failure” ratio for hardware components. Ali three strands were destined to survive, in modified forms, until the present, and they are still in use. In 1979, the Three Mile Island accident shook the nuclear-power world to its core, and interest in HRA surged. In 1981, the draft THERP handbook was made available by Swain & Guttman, and the use of THERP for PRAs/PSAs soared, THERP being the most effective approach at this time of crisis in the nuclear-power industry. In 1982, the first glimpses of a Human Cognitive Reliability (HCR) approach were beginning to appear (Hali et al, 1982); the HCR technique was arguably a logical descendant of the Askren & Regulinski approach of the 70s. This in turn led onto SHARP (systematic human-action-reliability procedure, Hannaman et al, 1984) which was the second total framework for HRAs, from task analysis right through to quantification. In 1984, the Success Likelihood Index Method Using Multi-Attribute Utility Decomposition (SLIM-MAUD) approach was published as part of a USNRC Nureg report (Embrey et al, 1984). This represented a psychologically dominated approach, as opposed to engineering- and reliability-based approaches such as THERP and HCR. Shortly after SLIM-MAUD came the SHERPA system (the systematic human-error reduction and prediction approach, Embrey, 1986) – the third “total” HRA framework, along with SHARP and THERP (although ali three were also heavily quantification-biased).
The Absolute Probability Judgement (APJ) and Paired Comparison (PC) techniques were also fully documented for use in HRAs and PRAs (Seaver & Stillwell, 1983). In 1986, the Human Error Assessment and Reduction technique appeared (HEART) which was heavily based in the ergonomics domain, and which actually encompassed error-reduction guidance.
The technique of Influence Diagrams, the first one capable of considering aspects such as safety culture, was also starting to be used in PRAs (Philips et al, 1983).
Although Three Mile Island was beginning to be written off as a one-off accident, and the need for HRAs was being questioned, the subsequent occurrence of accidents such as Bhopal (1984), and then Challenger and Chernobyl (both 1986) ensured that the HRA stayed firmly a part of the PSA process. There was now a flourishing range of techniques.
Moreover, the predominance of engineering approaches was now going into reverse, and greater credibility was being given to more psychological and expert-opinion-based quantification approaches.
In 1988, the Human Reliability Assessor’s Guide (HRAG document) (Kirwan et al, 1988) presented the first full, detailed peer review of HRA techniques, to be followed a year later by Swain’s own exhaustive review. At present, there are one or two new or hybrid techniques that involve unusual (from a historical perspective, at least) partnerships – for example, that between the SLIM and HCR (Bley et al, 1988). However, today, efforts are beginning to be directed away from quantitative HRAs and towards qualitative HRA insights instead. It can be seen from the above that, originally, HRAs attempted, somewhat unsuccessfully, to develop data-banks. Later on, HRA practitioners resigned themselves to using expert-judgement techniques such as the SLIM and Absolute Probability Indgement (APJ), or else techniques which were a mixture partly of data and partly of expert judgement (i.e. that of the technique’s author), such as THERP, HEART and, the latter, Human Reliability Management System (HRMS). Recently, however, there has been a renewed interest in the data-bank concept, most notably with the NUCLARR project (Gertman et al, 1988), as well as with a few others (e.g. Kirwan et al, 1990; Taylor-Adams & Kirwan, 1994). Currently, such data-banks are not themselves ready to be used for a direct HRQ, but their development rate is such that this situation may change in the near future.
In 2000, the study of this problems in Serbia began when a series of papers published (Savic and Grozdanovic, 2000; Grozdanovi, and Radojkovic 2001; Grozdanovic and Savic, 2001; Grozdanovi, et al 2002; Grozdanovi, et al 2005; Grozdanovic and Stojiljkovic, 2005; Stojiljkovi et al, 2005; Stojiljkovic and Grozdanovic, 2005).
2. HUMAN RELIABILITY MANAGEMENT SYSTEM
The following part describe the modules of the HRMS in terms of what functions they achieve and how broadly they work.
Representation module-the purpose of this module is to ensure that errors are repreented or described in a way which can be accurately quantified both by the quantification module (PHOENIX) and by the fault-event-tree methodology when integrated into the Probabilistic Safety Assessment (PSA) and evaluated within the logic-tree format.
The primary role of this module, however, is to determine which of the errors identified are to be represented and quantified in the assessment. Five categories exist which may be removed from further assessment at this stage: errors which have no consequence; errors already identified; errors which are incredible; errors with a very high (virtually certain) chance of recovery; and errors which may be ‘subsumed’ under other HEPs, i.e. aggregated within a higher-level datum. Quantification module-is based primarily on the best available data from actual historical measurements, from simulator studies and from experimental research, as well as on certain derived data.
However, it is unlikely that such a limited database would suffice for all safety cases.
Furthermore, such a database would yield little, if any, useful information on error reduction during an assessment. As a result, the quantification system also utilises six operationally defined Performance Shaping Factors, derived both from SLIM analyses carried out during various experiments, and from a review of several other techniques and assessments. Each of these PSFs involves various levels, and any scenario can be described by reference to a particular PSF. In addition, any scenario can also be described from the point of view of any one of the PSFs; thus, each datum has a particular profile for each of these PSFs, as well as an attached probability of human error.
Sensitivity module- from the description given in the above section on the quantification module, it is apparent that for any given error, the relative importance of different PSFs can be calculated. If an HEP is derived which is too high, then the sensitivity-analysis capability within HRMS will allow the user to see how much of a change in probability could be brought about by a modification of the degree of importance assigned to the various PSFs. This is one method of error-reduction analysis. Error reduction module-a set of guidelines are available on how to reduce errors of various types. These guidelines, influenced by the above named PSFs, are practically oriented, in terms of design and operational parameters, and are aimed at reducing the root causes of error, at increasing a system’s level of error tolerance, at enhancing error recovery or else at generally improving the standard of human performance. Guidelines also exist on how to feed error-reduction assumptions through the quantification process, and how, then, to ensure that they are implemented.
Documentation-the system is largely self-documenting, via print-outs, occurring at various stages in the program, which can be appended to safety cases and recorded in the HRMS ‘library’, as well as being stored on computer disk. The system also documents the user’s identity and details of the safety case, as well as relevant dates, etc. Quality assurance-all documented assumptions are passed on to respective design and operations departments and followed through until sanctioned and, ultimately, satisfactorily implemented. If one or more assumptions are not ”cleared” in this way, a reassessment of the safety case will be carried out.
Changes made to the operational design of the system during its operational lifetime can be checked to see if they affect any of the safety cases or safety-case assumptions. In addition, any incident information about an incident or incidents that is relevant to the safety cases can be analysed to determine whether the HEPs are accurate, and the HEPs can be adjusted if necessary. In this way, the database would become more realistic as the plant ages.
2.1. Management of Performances Shaping Factors Incidents caused by human errors may often lead to severe accidents. In order to prevent future human errors, not only the causes of human errors but also the seeds of these errors should be understood. In that respect, the management of PSF is needed. PSF is the factor that may cause human error incidents. Though several PSF studies have been done in human reliability analysis, only few pragmatic studies can be found out. In this paper, we introduce the PSF evaluation method that can be easily applied for many organizations, and the results in several case studies are shown. We called the system, which enhances the reduction of human error on PSFs, as human error management system. Performance shaping factors is the factors that affect human behaviour.
Human error is also one of human behaviour. Accordingly, PSFs in human error behaviour are regarded as causes of human error.
In order to prevent reoccurrence of human errors, PSFs that causes human error should be extracted exhaustively, and the extracted PSFs should be removed or improved. On the other hand, PSFs that exert bad effect on human behaviour can become a cause of human error. If PSFs can be improved, the possibility of human error occurrence would decrease. In other words, PSF analysis is contributed to preventing the future human error occurrence. In this case, PSFs can be regarded as seeds of human error.
Thus, PSF is very important key in human error. It can be said that human error management is to control PSFs in the organization. Category of PSFs based on human information processing model is shown in Figure 1.
In order to obtain PSFs in target human behaviour, the method that extracts PSFs is used. As PSFs have a hierarchical structure, PSFs that causes PSFs should be also investigated. From the PSFs structure whose summit is human error, the human error preven provement of PSFs.
Immediate PSFs: The improvement of immediate PSFs cannot be expected to prevent other trouble occurrence. The prevention strategy against the immediate PSFs tends to depend on the specification of object task. So the strategy cannot be applied for tasks in other departments, and possibility of occurrence of similarity troubles in other departments is not changed, that is, the possibility keeps high. Latent PSFs: As latent PSFs exist in other departments, the prevention strategy against the latent PSFs is expected to reduce the possibility of human error occurrence. Only if the information of the strategy is transmitted to other departments, the similarity troubles will be prevented. However, these methods only function as a support to describe the extracted PSFs. So, if the subject which analyzes the PSF has few experiences of PSF analysis, sufficient results would not be obtained.
In particular, if latent PSFs that are background factors of human error were not extracted, the effect of the planned prevention strategy would not be expected. For supporting extraction of latent PSFs, the following method, which has the reference such as PSF keyword table (Table 1) and frequency of PSFs (Figure 2) extracted from incident reports, is effective. There are a lot of cases that more than one human error exists in one trouble. So, with observing the operational sequence, the detail that leads the trouble should be cleared. An every human behaviour that related to the trouble should be analyzed on PSFs.
Thus, analyzing PSF and estimating the possibility of human error occurrence could produce the effective prevention strategy. However, as the usability of the method is not high, the computer system, which supports PSF analysis and proposes the guideline of prevention strategy, is now developing. It is desired that human error management progresses the following spiral flow. (Figure 3). Of course, the basic background of the management system is PSF. However, PSF should not be hated. It is important that we make friends with PSF. The quality of human behaviour would be advance, if PSF can be controlled well. In other words, usability and comfortableness would increase by PSF study.
3. HUMAN ERROR DATA
There are two major types of human-error data which can be collected:
Qualitative data: this information provides both general error-reduction strategies, based on human-factors experimentation, and also specific error-reduction guidelines, based on feedback from operational experience.
Quantitative data: this information can be in the form either of relative data, e.g. “the probability of error A is half that of error B”, or of absolute data, e.g. “the probability of error A is 0.1”.
Both types of data are useful in the context of human-reliability assessments, but there is in particular a need for the collection of absolute quantitative data for use in Probabilistic Risk Assessments (PRAs). These HEP estimates can then be used either in the validation of techniques which have been developed to quantify human error or more directly, for quantification purposes, if enough useful data exists.
Three potential sources exist for the collection of data suitable for the generation of HEPs. These are:
- Data derived from relevant operating experience,
- Data derived from experimental research,
- Data derived from simulator studies.
Ideally, all data collected would be taken from relevant operating experience, or from sufficiently robust and industrially relevant experiments. Unfortunately, very little data have been collected from such sources, and thus recourse has had to be made to data from other sources such as the judgement of experts. The main reason for this is that there are a number of serious difficulties associated with the collection of operational experience data; these are discussed below. Three major technical problems exist in relation to the generation of HEP data.
The first problem is concerned with the degree of specificity inherent in the data given for the plant undergoing a PRA. Large variations exist between different plants in terms of the way they are operated, their training and procedural facilities, the safety-management culture, the ergonomical level of adequacy inherent in the equipment’s design, etc. In PRAs, either of these types of data are likely to be applied indiscriminately to widely different types of plant.
However, there is a second and more immediate problem, which concerns the usefulness of the data for error reduction purposes. The types of data mentioned above do not give information on how to improve human reliability in those cases where it is found during the PRA evaluation that the plant is not satisfying the risk criteria (due to the human-error impact). In this case, if there is no easy way of improving human reliability, then other approaches (interlocks, automation, extra safety systems, etc.) may have to be considered.
A third problem with purely quantitative data, one related to the second problem, is that such data only state the external form, or observable manifestation, of the error (i.e. the external error mode – EEM). Returning to the above mentioned example of turning the valve the wrong way, this error could be due to a momentary aberration on the part of the operator called a “slip”. It could also occur as a result of the operator’s experience on other plants where valves had to be turned in the opposite direction. The point here is that the operator involved in the second situation is far more likely to make the error, due to a “population stereotype”, than he or she would if the error were due purely to a “slip”. And the associated HEP could also differ dramatically. The external error mode is the same but the actual root cause, or psychological error mechanism (PEM), is different.
Table 2, shows some data available from a range of different sources. These forms of information comprise: generic data, data from operational plants, data based on ergonomics studies and data from simulator studies. Table 3, shows a full set of EEMs and PMS, that can be utilised in human error analysis.
4. CONCLUSION
Human reliability quantification techniques all quantify the human error (HEP), which is the metric of human reliability assessment. The ideal source of human error data would be from industrial studies of performance and accidents. Other sources are simulator data and data derived in the human performance literature.
Therefore, there is a data problem. Such difficulty has led to the development of non-data-dependent approaches, namely to the use of expert opinion. This is by no means necessarily a bad thing, and expert opinion has been used successfully in the other areas, and is in any case used at least occaasionally in probabilistic safety assessment where similar problems often exist. Human error is not the personal work but one of the issues that the organization should solve. For that, the situation factors, the media factors, and the management factors as well as the personal characteristics must be investigated, and the environment where human error doesn’t happen easily will need to be provided.
First step in human error management is that human error is recognized as the organizational issue. It is desired that the worker considers what kinds of human error will be able to happen in present work circumstances. The ability makes the imagination of human error to be obtained by the experience of PSF analysis. As a worker, who has such ability, the increase in the organization, an effect of human error management improves.
The purpose of human error management is not the investigation of past cases but the improvement of the essence of the organization. That is, human error management is developed for the solution of the future problem on organizational management. Accordingly, the management system that more intelligence information can be acquired should be constructed. It is important that not only the facts but also the possibilities are evaluated in detail. The human error management must evolve in accordance with the change in the period and the society. A conclusion made in the (HRAG) document was that there were already techniques available for carrying out the quantification of most, if not all, types of scenario; but also that the practitioner needed to be flexible in selecting techniques appropriate to each situation. Whilst noting that there were techniques capable of carrying out HRAs, he lamented the lack of R&D funding in this field, and pointed out, furthermore, that until a proper database was set up and more evaluations were carried out, Human Reliability Assessments would continue to remain the least credible part of the Probabilistic Risk Assessment (PRA) process. It is interesting that Swain, whose THERP data-base is the most widely used data-bank in existence, is still arguing for the generation of real data as opposed to “synthetic” or “simulator” data and there is, in fact, a resurgence of interest in collecting usable data. There is also an increasing interest in the carrying-out of validations of HRQ techniques, both to see how accurate they are in practice and to allow the techniques themselves to be improved.
In the meantime, before databases are constructed and techniques have been fully validated, the practitioner does have plenty of tools from which to choose. It is hoped that these will aid the practitioner in selecting techniques for real applications.