Contemporary Ergonomics 2009
Contemporary Ergonomics 2009 Editor
Philip D. Bust Loughborough University
© 2009 Taylor & Francis The Development of HSL’s Safety Climate Tool – A Revision of the Health and Safety Climate Survey Tool S. Sugden, M. Marshall, S. Binch & D. Bottomley © Crown Copyright 2008/HSL Reproduced with permission of the Controller of her Britannic Majesty’s Stationary Office. The views expressed are those of the Author and do not necessarily reflect the views or policy of the Controller or any government department. The Controller, any government and Taylor & Francis Ltd accept no responsibility for the accuracy of any recipe, formula instruction contained within this population. Typeset by Macmillan Publishing Solutions, Chennai, India Printed and bound in Great Britain by Antony Rowe (A CPI-group Company), Chippenham, Wiltshire All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Every effort has been made to ensure that the advice and information in this book is true and accurate at the time of going to press. However, neither the publisher nor the authors can accept any legal responsibility or liability for any errors or omissions that may be made. In the case of drug administration, any medical procedure or the use of technical equipment mentioned within this book, you are strongly advised to consult the manufacturer’s guidelines. ISBN 978-0-415-80433-2 (Pbk) ISBN 978-0-203-87251-2 (eBook)
Contents Preface
XI
PLENARY SPEAKERS Inspection: From goods through passengers to systems C.G. Drury Business ergonomics beyond health and safety: Work environments for employee productivity, creativity and innovation J. Dul
3
16
Occupational safety: Charting the future Y.I. Noy
24
Ergonomics and public health P. Buckle
33
Human factors and development of next generation collaborative engineering J.R. Wilson, H. Patel & M. Pettitt
39
ACCESSIBILITY A design ergonomics approach to accessibility and user needs in transport R. Marshall, D.E. Gyi, K. Case, J.M. Porter, R.E. Sims, S.J. Summerskill & P.M. Davis Development of tools for reducing the stress of using public transport P.M. Davis, R. Marshall, K. Case, J.M. Porter, D.E. Gyi, S.J. Summerskill & R.E. Sims Validation of the HADRIAN system with a train station design case study S.J. Summerskill, R. Marshall, D.E. Gyi, J.M. Porter, K. Case, R.E. Sims & P.M. Davis
51
61
70
COMPLEX SYSTEMS A software tool for evaluating the effects of culture on military operations A. Hodgson & C.E. Siemieniuch V
81
VI
Contents
Decision-making after the product-service shift and some implications for ergonomics E.M. Molloy, M.A. Sinclair & C.E. Siemieniuch Governance, agility and wisdom in the capability paradigm M.A. Sinclair, M.J.D. Henshaw, R.A. Haslam, C.E. Siemieniuch, J.L. Evans & E.M. Molloy
91
100
DESIGN The role of ergonomics in technology oriented projects; the intelligent zero emissions vehicle project A. Woodcock & J. Taylor
111
Research and policy development into scooter access onto London buses S. Thomas
122
HAZARDS Study of verbal communication between nuclear plant control room operators during abnormal situations A.N. Anokhin & N.V. Pleshakova …It’s the way that you do it: Ensuring rule compliance J. Berman, P. Ackroyd & P. Leach
135
145
HEALTH ERGONOMICS AND PATIENT SAFETY Public requirements for patient-held records J. Binnersley, A. Woodcock, P. Kyriacou & L.M. Wallace
159
Systems analysis for infection control in acute hospitals P.E. Waterson
165
Portable pods: Design for unscheduled (urgent) care in the community S. Hignett, A. Jones & J. Benger
174
Using usability testing as a benchmarking tool – A case study on medical ventilators Y. Liu & A. Osvalder
182
Contents
Does the older workforce with high work demands need more recovery from work J.J. Devereux & L.W. Rydstedt
VII
189
HUMAN ADVISORY The role of the human in future systems – Considerations and concerns M.S. Young
199
Automation and technology in 21st century work and life S. Sharples
208
Vigilance and human supervisory control – A potted history J.M. Noyes
218
HUMAN ERROR/ACCIDENTS Unpacking the blunt end – Changing our view of organizational accidents B.M. Sherwood Jones Lessons from the past to give to the future: Learning from incidents S. Evans The development of HSL’s safety climate tool – A revision of the health and safety climate survey tool C. Sugden, M. Marshall, S. Binch & D. Bottomley
229
236
245
HUMAN FACTORS INTEGRATION Reporting location and environmental features when using metal detector or probe in a simplified mine detection task R.J. Houghton, C. Baber & J.F. Knight Predictive Operational Performance (PrOPer) model K.T. Smith Distributed situation awareness: Applications and implications in defence P.M. Salmon, N.A. Stanton, G.H. Walker & D.P. Jenkins
255
263
272
VIII
Contents
Human factors models of mini unmanned aerial systems in network-enabled capability C. Baber, M. Grandt & R.J. Houghton Human factors past and present W.I. Hamilton
282
291
Ergonomics challenges for digitisation of mission planning systems N.A. Stanton, G.H. Walker, D.P. Jenkins & P.M. Salmon
300
Human-centered design focus in systems engineering analysis: Human view methodology R.J. Smillie & H.A.H. Handley
310
Submarine safety and spatial awareness: The subsafe games-based training system R.J. Stone & A. Caird-Daley
320
Criteria for assessing social and organisational impacts of emerging technologies P. Sirett, R. Braithwaite, R. Cousens, J. Marsh & M. Mason
332
INSPECTION Animating the inanimate: Visual inspection from the medical to the security domains A. Gale
347
Inspecting for safety hazards K. Woodcock
354
Recent advances in technology to improve inspection training A. Gramopadhye, P. Stringfellow & S. Sadasivan
363
Using body discomfort correlations to assist in ergonomic inspections of computer users P. Stringfellow & A. Gramopadhye
373
One year later: How screener performance improves in X-ray luggage search with computer-based training A. Schwaninger & A.W.J. Wales
381
Contents
IX
METHODS AND TOOLS The hexagon-spindle model for ergonomics R. Benedyk & A. Woodcock
393
PRIMARY INDUSTRIES A biomechanics workload assessment of female milking parlour operatives M. Jakob, F. Liebers & S. Behrendt Swedish perspectives on ergonomics in the primary sector P. Lundqvist
405
414
ROAD ERGONOMICS A 50-driver naturalistic braking study: Overview and first results N. Gkikas, J.H. Richardson & J.R. Hill
423
Motorcycle ergonomics: Some key themes in research S. Robertson, A.W. Stedmon, P.D. Bust & D.M. Stedmon
432
Keeping it real or faking it: The trials and tribulations of real road studies and simulators in transport research A.W. Stedmon, M.S. Young & B. Hasseldine
442
SCHOOLS Pupil involvement in classroom (re)design: Participatory ergonomics in policy and practice A. Woodcock, J. Horton, O. den Besten, P. Kraft, M. Newman, P. Adey & M. Kinross Results from a post-occupancy evaluation of five primary schools M. Newman, A. Woodcock & P. Dunham
453
462
STRATEGIC ERGONOMICS Repositioning human factors/ergonomics (HF/E) – Raising high hazard industry understanding and commitment to HF/E D. Pennie, M. Wright & P. Mullins
473
X
Contents
Ergonomics for engineers P.J. Clarkson The trials and tribulations of trying to apply ergonomics in the real world M. Kenwright & D. Pennie
483
491
TRANSPORT A preliminary investigation of a low-cost method for prediciting the disruption of glances towards in-vehicle information systems A. Irune & G.E. Burnett
503
The effective management of user requirements and human factors issues C. Munro & S.R. Layton
513
WORKING PEOPLE Predicting the human response to an emergency G. Lawson, S. Sharples, S. Cobb & D. Clarke
525
Human factors contributions to the design of a security promotion campaign: A case study A.E. Peron, B.F. Davies & S.R. Layton
533
Author Index
543
Preface Contemporary Ergonomics 2009 is the proceedings of the Annual Conference of the Ergonomics Society, held in April 2009, at the Royal College of Physicians, London, UK. The conference is a major international event for Ergonomists and Human Factors Specialists, and attracts contributions from around the world. This year marks the 60th anniversary of the Ergonomics Society. Papers are chosen by a selection panel from abstracts submitted in the autumn of the previous year and the selected papers are subject to a peer review process before they are published in Contemporary Ergonomics. Each author is responsible for the presentation of their paper. Details of the submission procedure may be obtained from the Ergonomics Society. The Ergonomics Society is the professional body for ergonomists and human factors specialists based in the United Kingdom. It also attracts members throughout the world and is affiliated to the International Ergonomics Association. It provides recognition of competence of its members through its Professional Register. For further details contact: The Ergonomics Society Elms Court Elms Grove Loughborough Leicestershire LE11 1RG UK Tel: (+44) 1509 234 904 Fax: (+44) 1509 235 666 Email:
[email protected] Web page: http://www.ergonomics.org.uk
XI
PLENARY SPEAKERS
INSPECTION: FROM GOODS THROUGH PASSENGERS TO SYSTEMS Colin G. Drury University at Buffalo, SUNY, USA Ergonomics work on inspection started with field studies of people inspecting components repetitively, often in a line production environment. The original papers on the ergonomics of industrial inspection were often written by psychologists with a highly interpretive and contextual view of the process they were reporting on. Since that time (1950’s) studies moved to laboratory settings, with emphasis on mathematical modeling, although many quantitative field studies were also performed. Modern production processes deemphasize and even denigrate inspection as unproductive or counterproductive. Ergonomics studies of inspection began to expand into maintenance inspection, e.g. of aircraft or their components, into medical radiology and more recently into the airport security domain. With good qualitative and quantitative models of inspection available, we should now be in a position to tackle more abstract domains, applying what we have learned to inspection of organizations for safety, or complex paperwork systems for compliance. In these areas, what we know about inspection complements what we know about auditing processes and systems for compliance. There is a demand to better understand such inspection/audit activities, for example in government oversight of aviation maintenance or procedure compliance.
Introduction Inspection tasks have been a minor but continuing theme in human factors / ergonomics (HFE) practice and literature since the 1950’s. Many ergonomists have been asked to improve jobs and workplaces where items were inspected and have contributed to an evolving understanding of how people make repetitive decisions, often Yes/No decisions about goods and services. Most of our understanding of inspection tasks is based on these practical studies, often augmented by off-line studies to better model the effects of variables noted but uncontrolled in the field. Of course ergonomists were not the only ones studying inspection: there are long traditions in quality control and equipment/automation design that have very little (and that often derogatory) to say about the human in the inspection system. More modern strictures on quality have emphasized open-loop control of industrial processes rather than inspection as a way of ensuring that defective items are all but eliminated from the process. This has led to calls to end inspection a position 3
4
Colin G. Drury
significantly aided by automation of many measurement tools. A different class of inspection tasks is embodied in the continuing inspection of built systems to ensure that defects have not arisen and thus the system is still safe for public use. Here we include the inspection of railway tracks for a range of conditions that could compromise passenger safety, and parallel inspections of civil aircraft structures, or of passenger bags being brought into the civil aviation system. Without too much of a stretch, this class can include medical inspection, e.g. radiography or endoscopy, designed to ensure that the human body is still working as expected. In parallel to the quality control literature, there is an equally well-developed literature on auditing, although again this rarely makes explicit mention of the human auditor. There have been audit programs developed for ergonomics / human factors but little consideration of auditing as inspection of systems rather than of objects. Finally we consider the potential application of findings on human and automated inspection to audit tasks by comparing their functional similarity to ask whether we can improve auditing processes by treating them as inspection of systems. The genesis of this extension came from a request by G. Salvendy for the author to combine his previous Handbook chapters on Human Factors and Automation in Test and Inspection (Drury, 2001a) and Human Factors Audit (Drury, 2001b) into a single chapter Human Factors and Ergonomics Audits (Drury, 2006). The obvious response to this request was to devise a framework that would cover both traditions and lead to new applications. This paper is the result.
Inspection of product Inspection has been a traditional part of the whole production process with written roots dating back almost 5000 years to ancient Sumaria. In the epic Gilgamesh, the narrator invites the reader to examine the quality of the walls of Uruk, built by the King: “Take hold of the threshold stone–it dates from ancient times! Go close to the Eanna Temple, the residence of Ishtar, such as no later king or man ever equaled! Go up on the wall of Uruk and walk around, examine its foundation, inspect its brickwork thoroughly. Is not (even the core of ) the brick structure made of kiln-fired brick, and did not the Seven Sages themselves lay out its plans?” (Gilgamesh, Tablet 1) The essence of the examining function is all there already. Bodily senses (look, take hold of ) are used to compare the existing item (wall) with some implied or actual standard (e.g. kiln-dried brick). In traditional craft skills the functions of inspection were part of the overall job of the craftsperson, no different in essence from other task elements. With the advent of the industrial revolution, and its development of F. W. Taylor’s “Scientific
Inspection: From goods through passengers to systems
5
Management”, the breakdown of jobs into separate tasks performed by separate people led to specialized inspectors. Taylor himself (Taylor, 1911 Chapter 2) studied the inspection of ball bearings in bicycle manufacture, concentrating (as might be expected) on speed and throughput rather than accuracy, although he did report increases in overall quality as well as quantity after the job was thoroughly studied and rest breaks introduced. Also early in the 20th century the concepts of statistical quality control were being introduced, where specified accuracy was paramount and neither speed nor worker well-being was an explicit consideration. The linking of accuracy, throughput and well-being in industrial inspection tasks awaited the insights of psychologists (human factors, ergonomics) after World War II. In the 1950’s and 1960’s studies of industrial inspection were undertaken in the UK and USA for a variety of products from electronic wiring (Jamison, 1966), through ball bearings (Belbin, 1963) to newly-minted coins (Fox, 1964). They combined detailed observations and interviews with measurements of defect detection performance. Performance was generally found to be poorer than expected (e.g. Table 1 of Gallwey, 1998) and often quite variable between inspectors and from day to day. These early studies also found some of the factors affecting human performance in inspection tasks, such as the presence of comparison standards for wool grading (McKenzie, 1958). Early work by Jamieson (1966), Thomas and Seaborne (1961) and McKenzie (1958) established that social pressures are an important part of the inspector’s job, and that inspectors can at times change their behavior and performance in response to such pressures. These field studies have continued throughout the study of industrial inspection, e.g. with knitting machine needles (Drury and Sheehan, 1969), and a whole series of electronics components (Harris and Chaney, 1969). Interesting new ideas were brought in by Lloyd et al., 2000), using customer ratings of the importance of defects to devise lighting systems for car bodies that emphasized those most important to the customer. Industrial inspection studies continue (e.g. Dalton and Drury, 2004 on sheet steel inspection). Good reviews of many published studies and the factors found to affect inspection performance are found in Megaw (1979) and Gallwey (1998). Parallel to this field (and field-based) work, there has been a tradition of modeling of inspection behavior. At its simplest, this has involved a classification of the subactivities of inspection, typically in the form of a function or task description. A typical list is adapted from Drury (2006) below but many others are available: Setup, where the inspection system is prepared for use. Needed tools, equipment and supplies are procured and calibrated, procedures are available to aid the inspector, and the inspector has been trained to perform the task correctly. Present, where the inspector and the entity to be inspected come together so that inspection can take place. Search is typically a sequential serial process during which the whole item to be inspected is brought piece-by-piece under scrutiny, stopping when an indication is found or further search is considered unnecessary. Decision is where the indication located by search is judged against a standard to determine whether it is a true defect. Response, where the action chosen in Decision must be taken correctly.
6
Colin G. Drury
In the 1960’s and 1970’s the Signal Detection Theory (SDT) model became popular with inspection researchers, partly because both the simplest form of SDT and the simplest form of inspection have outcomes expressible as 2 × 2 contingency tables (e.g. Drury and Addison, 1973). The TSD model of detecting a signal of known characteristics within random noise was at least plausible for inspection, although a number of authors questioned the applicability of TSD’s assumptions when applied to inspection (e.g. Buck, 1975). In parallel to this TSD modeling, a number of papers used models of visual search and visual detection for inspection tasks (e.g. papers in Morris and Horne, 1960). This modeling theme was taken up in inspection of medical chest X-ray images (e.g. Kundell and LaFollette, 1972), see later. Visual search models assume that detection of a target only takes place after it is fixated by the eyes. They also assume some process by which successive eye fixations are located on the item searched, either as a random process, a systematic process or a systematic process with fallible memory for prior fixation locations (e.g. Morawski et al., 1980; Arani et al., 1984). In fact both the search process (e.g. visual search) and the decision process (e.g. TSD) are probably important in industrial inspection tasks. Both would imply some form of speed-accuracy trade-off, as was seen by Drury (1973) and later used in a generic inspection model (Drury, 1975). However, while this flow of field studies and models of inspection tasks was taking place, the whole idea of industrial inspection was being challenged by newer manufacturing techniques and philosophies.
The end of product inspection? With the introduction of more tightly coupled production systems, error cannot be tolerated without widespread consequences, so that control of quality at source is a key component of advanced manufacturing systems such as Flexible Manufacturing Systems (FMS) cells. The corresponding push towards Lean manufacturing has also made early control of errors a priority. The concept of controlling key variances at their source has arisen through the various iterations of quality movements, but is also a key feature of Socio-technical systems design. One major effect of this change in production architecture has been to renounce to a large extent post-production inspection and replace it by in-process inspection. Indeed, the third of Deming’s (1982) fourteen points was: Eliminate the need for mass inspection as the way of life to achieve quality by building quality into the product in the first place. Require statistical evidence of built in quality in both manufacturing and purchasing functions. With the ultra-low defect rates now being demanded and achieved, even highly effective sorting inspection leaves the error rate too high. If we require defective rates in parts per million (∼ 10−6 ), then even a hit rate on inspection of 99% (error rate ∼ 10−2 ) will not give the necessary result unless the production defective
Inspection: From goods through passengers to systems
7
rate is ∼ 10−4 . Even this level may be difficult with human inspectors as they tend to be less effective at defect detection as the overall quality improves (e.g. Panjawani and Drury, 2003). Thus we look instead for necessary precursors to error rather than error. If we know the mean of a process has shifted by a given amount, then we can deduce that defects will be more common, even if we find no defects. This is the principle of in-process quality control, often using control charts. Where does this leave inspection? Is it to be eliminated completely, to be used on rare occasions when things go wrong, or must it move on beyond the manufacturing domain? In part the last of these alternatives is nearest the truth, mainly because of automation and information technology in manufacturing.
Inspection in other domains: Food, medical images, safety and security If we consider inspection ergonomics moving beyond its original industrial quality control domain, we can treat the reduced importance of this original domain as a cause or as an irrelevance. Inspection ergonomics had to evolve, but it was doing so well before the manufacturing revolution. One obvious area was in medical imaging inspection from the original Kundel and LaFollette (1972) beginnings to sophisticated use of models and tackling of new problems (Chen et al., 2008; Nodine et al., 2002). Another area that received ergonomics attention well before JIT was farm product inspection, e.g. of chicken carcasses (Chapman and Sinclair, 1973) or sexing of day-old poultry (Biederman and Schiffar, 1987). This activity still continues in all food industries but has received little recent attention beyond schemes for automation. There is a need for inspection to ensure airworthiness of civil and military aircraft, where inspection detects cracks, corrosion, loose fasteners etc. so that they can be repaired before they pose a threat to safe operations. Human Factors issues in civil aircraft maintenance inspection has received considerable attention following the fuselage damage to an airliner in 1988 for which the cause was found to be inadequate inspection performance (e.g. Drury et al., 1990). Since then studies have involved equipment design, training (e.g. Gramopadhye et al., 1987), job aids (Wenner et al., 2003) and good practices guides (Drury, 2003). A major growth area of inspection research has been in aviation security. This ergonomics research started many years ago (Funke, 1979) but expanded greatly after US terrorist attacks were widely perceived as resulting from inspection failures. Although all security screening tasks appear to fit the task description given above (Drury, 2002), most ergonomic attention has been given to X-ray screening of carry-on or checked baggage (e.g. Gale et al., 2000; McCarley et al., 2004; Hsaio et al., 2008). Models current in inspection, e.g. TSD, Drury Search Plus Decision model, visual search models and visual perception models, have all found applicability in this domain. Other areas of inspection that are increasing are single-stream recycling, where each object type has to be sorted (e.g. Associated Press, 2008), fire safety inspection of buildings and inspection of food facilities for compliance with local regulations. None have received ergonomics attention, but should be amenable to
8
Colin G. Drury
the observation, analysis and modeling techniques derived from studies of industrial inspection. In fact, the inspection of organizations for safety or compliance represents perhaps the next domain for inspection research. The manufacturing changes over the past two decades which heralded the demise of product inspection have also had the effect of moving organizations and safety professional away from reliance on post incident measures (e.g. injuries, equipment damage) towards pre-incident measures. In these, we are looking for valid indications of precursors to the rare events that accidents and injuries should become in a well-run organization. This field of examining organizations for compliance to standards already has its own techniques and tradition, under the rubric of auditing.
The auditing tradition When we audit an entity, we perform an examination of it. Dictionaries typically emphasize official examinations of (financial) accounts, reflecting the accounting origin of the term. Accounting texts go further, e.g. “. . . testing and checking the records of an enterprise to be certain that acceptable policies and practices have been consistently followed” (Carson and Carlson, 1977, p. 2). As with inspecting, auditing is mentioned in antiquity, at least in current translations. “. . . . . . he who does not pay the fine annually shall owe ten times the sum, which the treasurer of the goddess shall exact; and if he fails in doing so, let him be answerable and give an account of the money at his audit.” (Plato: Laws Book VI) Returning to the definition of an audit, the emphasis is on checking, acceptable policies, and consistency, to provide a fair representation of the business for use by third parties. A typical audit by a Certified Public Accountant would comprise the following steps (from Koli, 1994): Diagnostic Investigation: description of the business and high lighting of areas requiring increased care and high risk. Test for Transaction: trace samples of transactions grouped by major area and evaluate. Test of Balances: analyze content. Formation of Opinion: communicate judgment in an audit report. Such a procedure can also form a logical basis for safety audits. The first step chooses the areas of study, the second samples the system, the third analyses these samples while the final step produces an audit report. These are one way to define the broad issues in a safety audit design: How to sample the system: How many samples and how these are distributed across the system?
Inspection: From goods through passengers to systems
9
What to sample: What specific factors (valid and reliable precursors of safety problems) are to be measured? How to evaluate the sample: What standards, good practices or principles to use for comparison? How to communicate the results: Techniques for summarizing the findings, how far can separate findings be combined? Given this long auditing tradition, and its known applicability to ergonomics / human factors audits (e.g. Drury, 2001b), is there a structural similarity between inspection and auditing from which we can gain insight into the audit process?
Auditing as inspection of systems The simplest structural congruence between inspection and safety audit tasks is based on the task list of inspection given above: Setup implies the design of a sensitive, valid and reliable audit tool, and the training of auditors in its use. Present, implies a sampling plan for which parts of the organization to audit, and access to the sampling points chosen. Search is both the sequential following of the sampling plan and the active search of each area samples to locate possible indications. Decision is the judgment of the indication located by search against auditing standards to determine whether it is a true precursor of a safely problem. Response is the preparation and presentation of a clear audit report to those who commissioned the audit. Note that these map quite simply onto the four points derived from auditing: Diagnostic Investigation comprises the Setup task, Test for Transaction comprises the Present and Search tasks, Test of Balances is Decision while Formation of Opinion is the Response task. The list of how the four audit steps can be applied to safety audits merely rounds out this correspondence. This structural similarity can help the audit process by suggesting similar issues found in inspection tasks that could also occur in safety audits. All of these issues have been found in field studies of inspection and in the models used to better understand that function. A non-comprehensive list of issues is presented below in approximately the order of the generic inspection task list. Validity is the extent to which a methodology measures the phenomenon of interest. Content validity is perhaps the simplest but least convincing measure. If each of the items of our measurement device displays the correct content, then validity is established. Theoretically, if we could list all of the possible measures of a phenomenon, content validity would describe how well our measurement device samples these possible measures. In practice, it is assessed by having experts in the field judge each item for how well its content represents the phenomenon studied. Concurrent (or predictive) validity has the most immediate practical impact. It measures empirically how well the output of the measurement device correlates with
10
Colin G. Drury
the phenomenon of interest. Of course, we must have an independent measure of the phenomenon of interest, which raises difficulties. Construct validity is concerned with inferences made from scores, evaluated by considering all empirical evidence and models. Thus a model may predict that one of the variables being measured should have a particular relationship to another variable not in the measurement device. Confirming this relationship empirically would help validate the particular construct underlying our measured variable. The issue of validity of ergonomics audits has been addressed by many authors (e.g. Keyserling et al., 1993; Drury 2001b)) Validity for product inspection is typically less of a problem as the defects the inspection system is designed to prevent are usually based on customer input (e.g. Lloyd et al., 2000). However, even for products, validity can be compromised by the tendency for management to add indications to the list, so that in one example, inspectors were expected to detect over 80 different defects on the moving sheet steel (Dalton and Drury, 2004). Training has been seen as an important factor in inspection performance since the days of Belbin (1963). Clearly an untrained inspector is practically worthless, but the typical on-the-job training still found in many places is also highly inefficient. For example in X-ray security inspection (e.g. Schwaninger and Hofer, 2004) and in product inspection (Kleiner and Drury, 1992) a training scheme designed around psychological principles produced large performance improvements for even experience inspectors. For audit programs, the issue of providing effective training for auditors has not been addressed by ergonomists, although clearly safety auditors must be trained effectively. In industry there is typically a great respect for auditors with experience, but as noted above, training and experience may have different effects on performance. Accessibility is not typically an issue in product inspection but is a serious one for aviation maintenance inspection (Mozrall et al., 2000), where constricted spaces force awkward postures on inspectors of airframes and engines. A number of studies (Mozrall et al., 2000; Hsaio et al., 2008) have shown minimal effects of working posture on inspection performance, but postural discomfort is still suspected to be an ergonomic cause of poor performance in general. For audits, the equivalent issue is less one of posture than of permission to access. If an organization expects an audit then it can take steps to put on a good showing for the auditors. For this reason, most safety audits should be (are are) unannounced. It is still possible for organizations to hide problems (c.f. security inspection for threats) and to deny access to certain parts of buildings, so that auditors need to be aware of such potential biases in drawing up a sampling plan and in following logical indications when performing the audit. Access and Search overlap somewhat in safety audits, as they do for any other procedural inspection task. Speed-accuracy trade-off (SATO) has already been well-documented for product inspection and is clearly an issue for safety audits. Most of any SATO effect in product inspection appears to arise from the Search task (Drury, 1975). This results in an increase in probability of detection of a true defect and in probability of a false alarm as time taken per item increases (e.g. Ghylin et al., 2006, for security inspection). In audits the SATO issue has been characterized by Drury (2001b) as a balance between breadth, depth and application time. Ideally, an audit system would
Inspection: From goods through passengers to systems
11
be broad enough to cover any task in any industry, it would provide highly detailed analysis and recommendations, and would be applied rapidly. Unfortunately, the three variables of breadth, depth and application time are likely to trade off in a practical system. At the level of audit job aids, there are comprehensive ergonomics surveys such as the Position Analysis Questionnaire (McCormick, 1979) which take 2–3 hours to complete. Alternatively, there are simple single page checklists such as the Ergonomics-Working Position-Sitting Checklist (SHARE, 1990) that can be completed in a few minutes. The simplest job aid for any procedural task is the checklist. Safety inspectors in industry typically use a written checklist of several pages. Most checklists are used as memory aids for well-practiced tasks, so that they are structured as lists of commands, each of which is relatively terse: “switch both magnetos on: open fuel cock: prime engine for 5 seconds, etc.”. The user is expected to know the objects referenced and why each action is required. Checklists have their limitations, though. The cogent arguments put forward by Easterby (1967) provide a good early summary of these limitations in the context of design checklists and most are still valid today. Easterby quotes Miller (1967): “I still find that many people who should know better seem to expect magic from analytic and descriptive procedures. They expect that formats can be filled in by dunces and lead to inspired insights. . .We should find opportunity to exorcise this nonsense.” (Easterby, p 554) Easterby finds checklists can have a helpful structure, but often have vague questions, make non-specified assumptions, and lack quantitative detail. Checklists are seen as appropriate to aid operators (not ergonomists) in following procedural steps. Many checklists are developed, and many of these published, that contain design elements fully justifying such criticisms. Much research on checklists since that time has established a number of conclusions, all appropriate to safety audit checklists: 1. Checklist design: Use a “geographical” sequence of steps and good formatting/typography (Degani and Wiener; 1990), provide context information as well as instructions (Ockerman and Pritchett, 2000), use computer-based checklists as a more powerful alternative to paper-based ones (Larock and Drury, 2003) 2. People often use checklists in a manner not intended by designers, following their own sequence (Patel et al., 1993), using the object checked (an aircraft) for spatial cues (Pearl and Drury; 1995) or not using the checklist at all (Ockerman and Pritchett, 2000) Checklists are not the only issue in SATO for auditing as there is also the issue of time spent in the search phase of the task for any given design of audit procedure. Auditors, like inspectors, can be more diligent or less in their work. Decision Bias has been well-documented in inspection, typically in terms of the trade-off between missed defects and false alarms for product or security inspection. In auditing, there is a similar issue of potential bias in what is judged as an important
12
Colin G. Drury
finding and what is reported to the sponsor. Not all of the facts uncovered in an incident investigation are used in the report (Drury et al., 2003) and there is no reason to suspect that audits are immune to such selection biases. As Drury (2001b) notes, the design of the audit instrument can affect such biases by providing standards (Illumination must be >500 Lux) that can affect how the data is collected, rather than collecting the raw data (Record Illumination) and letting a computer program make the comparison to standards. Overall, a number of issues have been raised by this mapping of audits onto inspection tasks, all of which use inspection findings in the context of a safety audit. There are far more than can be pursued in future treatments, for example does a culture that encourages reporting and discussion of minor safety incidents (e.g. paper cuts) actually reduce the incidence of rare, large events?
References Arani, T., Karwan, M. H. and Drury, C. G. (1984). A Variable-Memory Model of Visual Search. Human Factors, 26, 63l–639. Associated Press (2008). Recycling innovation could end household ‘separation anxiety’ Wednesday April 30. Belbin, R. M. (1963). Inspection and Human Efficiency. Ergonomics in Industry Series, DSIR, London. Biederman, I. and Schiffar, M.M. (1987). Sexing day-old chicks: A case study and expert systems analysis of a difficult perceptual learning task. Journal of Experimental Psychology, Learning, Memory and Cognition, 13, 640–645. Buck, J. R. 1975, Dynamic visual inspection, in C. G. Drury and J. G. Fox (eds), Human Reliability in Quality Control (London: Taylor & Francis). Carson, A. B. and Carlson, A. E. (1977). Secretarial Accounting, (10th Edition) (Cincinnati, Ohio: South Western Publishing Company). Chapman, D. E. and Sinclair, M. A. (1975). Ergonomics in inspection tasks in the food industry. In C. G. Drury and J. G. Fox (Eds), Human Reliability in Quality Control, London: Taylor & Francis, 231–252. Chen, Y., Gale, A. and Scott, H. (2008) Anytime, Anywhere Mammographic Interpretation Training, Contemporary Ergonomics 2008, P. D. Bust (Ed.). Taylor & Francis, London, 375–380. Dalton, J. and Drury, C. G. (2004). Inspectors’ performance and understanding in sheet steel inspection. Occupational Ergonomics, 4, 51–65. Degani, A. and Wiener, E. L. (1990). Human Factors of Flight-Deck Checklists: The Normal Checklist, NASA Contractor Report 177548 (CA: Ames Research Center). Deming, W.E., (1982). Out of the Crisis, MIT, Cambridge, MA. Drury, C.G. (l973). The Effect of Speed of Working on Industrial Inspection Accuracy. Applied Ergonomics, 4, 2–7. Drury, C.G. (l975). Inspection of Sheet Materials – Model and Data. Human Factors, l7, 257–265.
Inspection: From goods through passengers to systems
13
Drury, C. G. (2001a). Human Factors and Automation in Test and Inspection, In G. Salvendy, Handbook of Industrial Engineering, Third Edition, Chapter 71, John Wiley & Sons, New York, 1887–1920. Drury, C. G. (2001b). Human Factors Audit, In G. Salvendy (ed), Handbook of Industrial Engineering. Third Edition, Chapter 42, John Wiley & Sons, New York, 1131–1155. Drury, C. G. (2002). A Unified Model of Security Inspection, Proc. Aviation Security Technical Symposium, Federal Aviation Administration, Atlantic City, NJ. Drury, C. G. (2003). Human factors and inspection: Best practices. Proceedings of the XVth Triennial Congress of the International Ergonomics Association, Ergonomics Society of Korea/Japan, Seoul/Korea, 584–587. Drury, C. G. (2006). Human Factors and Ergonomics Audits, In G. Salvendy (ed), Handbook of Human Factors and Ergonomics. Third Edition, Chapter 42, John Wiley & Sons, New York, 1106–1132. Drury, C.G. and Addison, J.L. (l973). An Industrial Study of the Effects of Feedback and Fault Density on Inspection Performance. Ergonomics, l6, l59–l69. Drury, C. G., Ma, J. and Woodcock, K. (2003). Do job aids help incident investigation? Contemporary Ergonomics, Proc. of the Ergonomics Society Annual Conference 2003, Edinburgh Scotland, April 15–17, 2003, Taylor & Francis, London. Drury, C. G., Prabhu, P. and Gramopadhye, A. (1990). Task analysis of aircraft inspection activities: methods and findings. Proceedings of the Human Factors Society 34th Annual Conference, Human Factors and Ergonomics Society, 1181–1185. Drury, C.G., and Sheehan, J.J. (l969). Ergonomic and Economic Factors in an Industrial Inspection Task. Int. Jnl. Prod. Res., 7, 333–34l. Easterby, R. S. (1967). Ergonomics checklists: an appraisal. Ergonomics 1967, 10, 548–556. Fox, J.G., 1964. The ergonomics of coin inspection. Quality Engineer 28, 165–169. Funke, D.J. (1979). Training: a significant factor in the effectiveness of airport baggage inspectors. Security Management, pp. 30–34. Gale, A.G., Mugglestone, M.D., Purdy, K.J., & McClumpha, A. (2000). Is airport baggage inspection just another medical image? Proceedings of the SPIE, 3981, 184–192. Gallwey, T. J. (1998). Evaluation and control of industrial inspection: Part II: The scientific basis for the guide International Journal of Industrial Ergonomics 22, 51–65. Ghylin, K.M., Drury, C.G., and Schwaninger, A. 2006, Two-component model of security inspection: application and findings, 16th World Congress of Ergonomics, IEA 2006, Maastricht, The Netherlands, July, 10–14. Gramopadhye, A. K., Drury, C. G. and Prabhu, P. (1997). Training for aircraft visual inspection, Human Factors and Ergonomics in Manufacturing, 3, 171–196. Harris, D. H., & Chaney, F. B. (1969). Human Factors in Quality Assurance. New York: Wiley.
14
Colin G. Drury
Hsiao, Y-L, Joseph, C., Joshi, S., Lapp, J., Pennathur, P. R, and Drury, C. G., (2008). Posture and performance: sitting vs. standing for security screening, Ergonomics, 51, 290–307. Jamieson, G. H. (1966), Inspection in the telecommunications industry: A field study of age and other performance variables. Ergonomics, 9, 297–303. Keyserling, W. M., Brouwer, M. and Silverstein, B. A. (1992). A checklist for evaluation ergonomic risk factors resulting from awkward postures of the legs, truck and neck, International Journal of Industrial Ergonomics, 9, 283–301. Kleiner, B. M., and Drury, C. G. (1992). Design and Evaluation of an Inspection Training Program. Applied Ergonomics, 24, 75–82. Koli, S. T. (1994). Ergonomic Audit for Non-Repetitive Task. Unpublished M.S. Thesis. State University of New York at Buffalo. Kundel, H.L., & La Follette, P.S. (1972). Visual search patterns and experience with radiological images. Radiology, 103, 523–528. Larock, B. and Drury, C. G. (2003). Repetitive inspection with checklists: design and performance. Proc. Ergonomics SocietyAnnual Conference 2003, Edinburgh Scotland, April 15–17, 2003. Lloyd, C. J., Boyce, P., Ferzacca, N., Eklund, N, and He, Y. (2000). Paint inspection lighting: Optimization of lamp width and spacing. Journal of the Illuminating Engineering Society, 28, 99–102. McCarley, J. S., Kramer, A.F., Wickens, C.D., Vidoni, E.D., & Boot, W.R. (2004). Visual Skills in Airport Security Screening. Psychological Science, 15, 302–306. McCormick, E. J. (1979). Job Analysis: Methods and Applications, (New York: AMACOM). McKenzie, R. M. (1958). On the accuracy of inspectors. Ergonomics, 1(3), 258–272. Megaw, E.D. (1979). Factors affecting visual inspection accuracy. Applied Ergonomics, 10, 27–32. Miller, R. B. (1967). Task taxonomy: science or art. In W. T. Singleton, R. S. Easterby and D. Whitfield (eds.), The Human Operator in Complex Systems. (London: Taylor and Francis). Morawski, T., Drury, C. G. and Karwan, M. H. (l980). Predicting Search Performance for Multiple Targets. Human Factors, 22, 707–718. Morris, A. and Horne, P. E. (eds) (1960), Visual Search Techniques (Washington, D.C.: National Research Council). Mozrall, J. R., Drury, C. G., Sharit, J. and Cerny, F. (2000). The effects of whole-body restriction on inspection performance. Ergonomics, 43, 1805–1823. Nodine, C.F., Mello-Thoms, C., Kundel, H.L., & Weinstein, S.P. (2002). Time course of perception and decision making during mammographic interpretation. American Journal of Roentgenology, 179, 917–923. Ockerman, J. J. and Pritchett, A. R. (2000a). Reducing over-reliance on taskguidance systems. In Proceedings of the Human Factors and Ergonomics Society 44th Annual Meeting, San Diego, CA.
Inspection: From goods through passengers to systems
15
Panjawani and Drury, C. G. (2003). Effective interventions in rare event inspection. Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting, Denver, CO, October 2003. Patel, S., Drury, C. G. and Prabhu, P. (1993). Design and usability evaluation of work control documentation, Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, Seattle, WA, pp.1156–1160. Pearl, A. and Drury, C. G. (1995). Improving the reliability of maintenance checklists. Human Factors in Aviation Maintenance – Phase Four, Progress Report, DOT/FAA/AM-93/xx, (Springfield, VA: National Technical Information Service). Schwaninger, A. and Hofer, F. (2004). Evaluation of CBT for increasing threat detection performance in X-Ray Screening. In K. Morgan and M.J. Spector (Eds). The Internet. Society: Advances in Learning, Commerce & Security, (pp 147–156). WIT Press.SHARE (1990). Inspecting the Workplace, SHARE Information Booklet (Australia: Occupational Health and Safety Authority). Taylor, F. W. (1911). The Principles of Scientific Management, Harper & Brothers, New York. Thomas, L. F. and Seaborne, A. E. M. (1961). The sociotechnical context of industrial inspection, Occupational Psychology, 35, 36–43. Wenner, C. A., Drury, C. G. and Spencer, F. W. (2003). The impact of instructions on aircraft visual inspection performance: A first look at the overall results. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Denver, CO, October 2003.
BUSINESS ERGONOMICS BEYOND HEALTH AND SAFETY: WORK ENVIRONMENTS FOR EMPLOYEE PRODUCTIVITY, CREATIVITY AND INNOVATION Jan Dul Rotterdam School of Management, Erasmus University, The Netherlands The value of ergonomics for companies is beyond health and safety. With ergonomically designed work environments a company can reach a competitive advantage. Ergonomics can help to improve employee productivity (especially in organisations focusing on a cost reductions), and increase employee creativity (especially in organisations focusing on innovation). Hence, ergonomics can have a strategic value for business organisations. In this paper, models and examples are presented showing that ergonomics can create business value. The consequences for ergonomics researchers and practitioners are mentioned.
Introduction According to the formal description of ergonomics, approved by the International Ergonomics Association, ergonomics deals with the design of products and processes in order to improve ‘human well-being’as well as ‘total system performance’. Improvement of human well-being can be considered as the ‘health, safety and comfort’ goal of ergonomics, which is important for the users of products and processes (consumers and workers). Similarly, improvement of overall total performance -although this term is rather vague-, can be considered as the ‘business’ goal of ergonomics, which is important for the management of an organization that develops, produces or uses products. This paper elaborates on the business dimension of ergonomics. Models and examples are presented which show how ergonomics can contribute to business goals. First I present a model how ‘product ergonomics’ and ‘production ergonomics’ can be linked to the common ‘value chain’ business model. Next, for production ergonomics, I will discuss to possible business topics for ergonomics: how to design work environments for human productivity, and how to design work environments for human creativity for innovation.
Business ergonomics In management, a business process is usually described as a chain of valueadding activities (Porter, 1985). The upper part of Figure 1 shows the 16
Business ergonomics beyond health and safety
Research
Product Process Purchasing development development
Production
17
Distribution
Product creation and realisation Product ergonomics Ergo-strategy Manager Ergo-innovation
Production ergonomics
Differentiation strategy with user-friendly products
Cost strategy with productive, human-friendly processes
Ergonomic Product development
Ergonomic process development
Ergonomic products
Ergonomic processes
Designer Ergo-system User
Figure 1.
Ergonomics as part of the value chain of regarding the ‘product’ and the ‘production’ (From Dul, 2003).
chain: ‘research, product development, process development, purchasing, production, and distribution’. The first part of the chain focuses on the ‘product’ and describes activities that must result in effective (in terms of sales) new or improved products or services. The second part of the chain focuses ‘production’and describes activities that must result in efficient (in terms of costs) realization of the product or service. Also in ergonomics a distinction has been made between ‘product ergonomics’ and ‘production ergonomics’. The lower part of Figure 1 suggests that product ergonomics can add additional value to the process of product creation (research, product development), and that production ergonomics can add additional value to the process of product realization (process development, purchasing, production, and distribution). With product ergonomics a competitive advantage can be achieved, by developing products that fit or exceed the functional and affective expectations of the user. For this strategic reason the management (the marketing manager, for example) can decide to use ergonomics. With production ergonomics, effective and efficient production systems can be realized using the capabilities of motivated, productive and creative employees. For this strategic reason the management (the operations manager, for example) can decide to use ergonomics. At the same time more acceptable working conditions regarding health and safety can be reached, which usually is a secondary argument for business ergonomics. In the remainder of this paper the focus will be on two business issues that are relevant to production ergonomics: productivity and innovation, by discussing how to design work environments for human productivity, and how to design work environments for human creativity for innovation.
18
Jan Dul
Work environments for productivity Since the industrial revolution, organizations focus on productivity: maximization of output at minimum cost. To support this objective, management principles and instruments have been developed and used in manufacturing and services companies all over the world. As a result, labour has been divided into small specialized tasks, processes have been standardized and subsequently mechanized and automated, and workers are specialized to do remaining tasks, many times resulting in repetitive and monotonic work with standard workplaces. However, when high levels of mechanization and automation have already been realized, further efforts to mechanize and automate the operations may not be as effective (ELA, 2004). In addition to ergonomics programmes that have been developed and implemented in order to reduce the negative side effects such work environments in terms 1 of health and safety and motivation, there is an increased attention for an ergonomics approach that combines productivity and health and safety. When ergonomics is applied, work environments can combine more efficient human movements, easier to assembly products, fewer rejected products etc., with less tiredness, better motivation, and improved health and safety. Ergonomically designed work environments may contribute to a reduction of sickness absenteeism and presenteeism. Presenteeism refers to those productivity losses occurring before an operator goes on sickness absence (Meerding et al., 2005). With ergonomically designed work environments fewer working days may be lost, costs to assist sick workers may be lower, return to work may be easier and quicker, there may be fewer cases of disability, lower personnel turnover, and fewer temporary workers to replace sick workers. In the ergonomics literature there is and increasing interest in productivity issues, and attempts have been made to provide empirical evidence of the productivity and costs effects of ergonomics programmes. For example, Koningsveld et al., 2005 analyzed 9 cases of ergonomics improvement projects, and found that the benefits from performance improvements vastly exceeded the benefits from reduced sickness absence alone. Several studies in the manufacturing industry in different countries have shown that with ergonomics productivity improvements can be realized (in combination with improved health and safety conditions). For example, Looze et al. (2003) analysed two cases in the Netherlands (one assembling office furniture, and the other assembling, testing and packing valves) report that with ergonomically improved technical and organizational processes (layout, task design, and work place design), productivity gains of 15–20% (and substantial reductions of time spent in risky body postures) can be achieved. Lee (2005), found in a Korean electric appliance company, that applying low cost ergonomics improved workplaces resulted in 10–30% productivity increase and US$17.01 million cost savings (and reduced number of musculoskeletal problems). Sen and Yeow (2003) studied the ergonomic redesign of electronic motherboard in a computer peripheral manufacturing factory in Malaysia, and found US $581,495/year cost savings and improved operators’ health and safety. These and other studies demonstrate that with ergonomically designed work environments considerable productivity gains can be achieved when applying ergonomics in the design of the production
Business ergonomics beyond health and safety
19
environments and that companies have indeed implemented this approach successfully.
Example Rhijn et al. (2005) studied work environment improvements of an assembly process in a Dutch company Famostar Emergency Lighting BV that develops, assembles and sells emergency lighting. The company is market leader The Netherlands. It produces 200.000 systems each year and is rapidly growing. In the old assembly process, a batch of parts were laid down on a table, and assembled and packed manually by workers who walked along the table. Finished products were placed manually on a pallet for further transportation. Due to an increase of the product volume and due to lack of space, a project was started to redesign the assembling process. The batch-type of production was changed into a flow-type of production. Sitting workplaces were introduced, which allowed picking parts from boxes close to the body, and lifting equipment was introduced to reduce manual lifting. The project showed the following results: labour productivity in terms of average number of products per person per day increased 44%, the order lead time reduced 46%, the space requirement decreased 44%, and manual lifting load reduced from unacceptable according to the NIOSH guidelines, to acceptable. However, the arm posture worsened because the workers worked 12% longer with a 20–60 degrees elevated arm posture. Overall, the results show that the assembling process was more productive and more human-friendly, although some nuances were found. This example shows how ergonomics can contribute the company’s productivity and other (strategic) business goals.
Work environments for creativity and innovation While most above examples are from manufacturing, in particular the economies of the Western world largely depend on services and knowledge work. In a knowledge based and innovation driven competitive business environment, work environments designed for productivity may not be the right choice. A company that needs to compete on innovation needs its organizational members not only for reaching productivity goals, but also for developing novel and useful ideas for solving problems and developing new products, services, processes, systems, work methods, etc. The creative potential of the workforce needs to be stimulated and work environments need to be designed to support worker creativity. Hence, organizations need workers – both employees at any level in the organization, and managers (Ceylan et al., 2008) – that can produce novel and useful ideas. Creativity has become a fundamental resource for many western organizations to maintain competitive. Organizations that pay attention to worker creativity normally focus on the worker, not on the work environment (Ceylan and Dul 2007). These organizations select creative workers, and provide creativity training. However, generating creativity by selection and training of the creative workers may not be sufficient. Creative workers that are placed in traditional productivity driven work environments may not show the desired creative behaviour. Organizational work
20
Jan Dul
Organizational Environment (e.g. job design, leadership)
(e.g. personality, cognitive style)
Creativity
Novel and useful ideas
Innovation
Individual
Physical Environment (e.g. workplace design, building design)
Figure 2. The effect of individual factors, organizational environment and physical environment on creativity and innovation (Adapted from Dul and Ceylan, 2006). environments that are designed for productivity, having formal management structures, time constraints, strict regulations, daily similar tasks, etc., may even obstruct or inhibit worker creativity. Also physical work environments that are designed for productivity, e.g. having standardised workplaces, may not support creativity. Dul and Ceylan (2006) have suggested that the ergonomics field could contribute to ‘bottom up’ innovation by studying and developing work environments that can support creativity and innovation. Based on empirical evidence from different fields like organizational behaviour, environmental psychology, business administration architecture, etc. they developed a conceptual framework called the ‘Creativity Development model’ (CD model) (Figure 2) on the relationship between individual characteristics, organizational work environment characteristics, physical work environment characteristics and individual creativity and related innovation. In comparison to existing models from the management literature the CD model explicitly highlights the role of the physical environment to support creativity. Based on this model Dul et al. (2007) developed the Creativity Development Quick Scan (CDQS) which is a checklist for measuring the creativity potential of work environments. The checklist includes 21 factors that could foster or hinder creativity. Four factors relate to job design for supporting creativity (challenging job, team work, task rotation, autonomy in job), 5 factors relate to leadership for supporting creativity (coaching supervisor, time for thinking, creative goals, recognition of creative ideas, and incentives for creative results), 5 factors relate to interior design for supporting creativity (furniture, indoor plants/flowers, calming colours, inspiring colours, privacy) and 7 items factors relate to building design for supporting creativity (window view to natural elements, any window view, quantity
Business ergonomics beyond health and safety
21
of light, daylight, indoor (physical) climate, sound, and smell). The checklist has to be filled in by the employee; for each factor the checklist measures on a 7-point scale the worker’s perceived climate for creativity. The CD Quick Scan has been applied in several studies, which resulted in a database that currently contains data from more than 1000 individuals with various professional backgrounds in 5 nations from different regions of the world. If individuals work in the same organisation the individual data can be aggregated to obtain the climate for creativity of an organization. Presently data are available from 30 companies. Many problems and solutions regarding work environments for creativity are very similar to the problems and solutions for work environments for comfort, health and safety. Hence, existing ergonomics knowledge could be readily applied. For other problems the required knowledge may not be readily available (yet) in the current ergonomics domain, although it may be available in other domains (organizational behaviour, business administration, environmental psychology, architecture, etc.). Yet for other problems further studies are needed to develop solutions.
Example A Dutch consultancy firm intended to stimulate entrepreneurship, curiosity and creativity of its consultants in order to enter new markets with existing competencies. First a CDQS analysis of the existing work environment in the company was performed, and the results were benchmarked against 4 other comparable organizations in the Netherlands. Tables 1 and 2 show the results of this analysis; the consultancy firm of interest is company 7. It turns out that the company scored relatively low for both the organizational climate for creativity and the physical climate for creativity. For the organizational climate for creativity, only ‘autonomy in the job’ scored higher than average. For the physical climate for creativity only ‘any window view’ scored higher than average. This comparison convinced the management that there was a need to make work environment improvements. First the results were discussed with the employees (the consultants), in order to get suggestions for improvements and prioritize them. With respect to the organizational work environment, there appeared to be a need for more teamwork (e.g. more consultants per project) for better exchanging information and ideas between consultants, and for realizing social support between consultants. There was a need for a more supportive and coaching rather than controlling leadership styles of the managers in order to stimulate creativity of the employees (e.g. by training of managers in leadership styles). With respect to the physical environment, several suggestions were made to change the workplace, which can be realized relatively easy (furniture, plants, colour). Currently, the suggestions are being considered by the management for implementation. One main result of this process was that on the basis of objective data, discussions started between management and consultants about the present situations and possibilities for improvements, similarly to any other participatory ergonomics approach.
22
Jan Dul
Table 1.
Organizational climate for creativity of 5 companies.
Company
2
4
5
6
7
All
Challenging Job Teamwork Task rotation Autonomy in job Coaching supervisor Time for thinking Creative goals Recognition of creative ideas Intensives of creative results Total Organisational
6.13 6.41 4.69 6.13 6.41 4.69 6.18 5.76 4.18 5.62
4.59 5.22 3.24 6.00 5.00 4.24 4.47 4.71 4.12 4.62
5.01 4.68 3.43 5.62 3.51 3.59 3.65 3.76 3.58 4.09
5.43 5.57 4.71 5.93 5.36 4.14 4.71 5.43 4.79 5.12
4.75 4.08 3.67 6.25 3.25 3.50 3.58 3.67 3.75 4.06
5.18 5.19 3.95 5.99 4.71 4.03 4.52 4.67 4.08 4.70
Table 2.
Physical climate for creativity of 5 companies.
Company
2
4
5
6
7
All
Furniture Indoor plants/flowers Calming colours Inspiring colours Privacy Window view to nature Any window view Quantity of light Daylight Indoor (physical) climate Sound Smell Total Physical
4.53 5.18 4.71 3.47 3.76 3.59 5.35 5.82 5.88 4.59 4.18 3.88 4.58
4.88 3.82 3.59 3.00 5.06 4.00 5.06 5.18 5.65 4.00 4.24 3.82 4.43
3.86 3.38 3.00 2.59 4.32 4.15 5.11 5.14 5.38 3.64 4.32 3.71 4.05
4.50 3.93 3.14 2.71 3.57 3.64 3.54 4.71 4.43 4.21 3.50 3.86 3.81
3.67 2.42 3.00 2.42 3.33 3.17 5.09 3.67 2.42 3.00 2.42 3.33 3.16
4.29 3.75 3.49 2.84 4.01 3.71 4.83 4.90 4.75 3.89 3.73 3.72 3.99
This example shows how ergonomics can contribute a company’s creativity and innovation policy.
Conclusion The above models and examples show that the field of ergonomics can add value beyond health and safety. Ergonomics can contribute to a company’s competitive advantage by providing approaches and solutions to improve labour productivity, or to improve the climate for creativity and bottom up innovation. However, most organizations are not aware of such value of ergonomics, and the same may apply to many ergonomists. Therefore in order to further exploit this added value of ergonomics in the future, ergonomists need to develop their skills and knowledge to link ergonomics to business strategies and goals, and to communicate this towards the management and business community (Dul and Neumann, in press).
Business ergonomics beyond health and safety
23
References Ceylan, C. and Dul, J. 2007. Expert opinions on human resources practices for creativity stimulating work environments for innovation in Turkey, the Netherlands, and Brazil. Paper presented at the 9th International HRM Conference on Changes in Society, Changes in Organizations, and the Changing Role of HRM: Managing International Human Resources in a Complex World, Tallinn, Estonia, June 12–15. Ceylan, C., Dul, J. and Aytac, S. 2008. Can the office environment stimulate a manager’s creativity? Human Factors and Ergonomics in Manufacturing 18(6), 589–602. Dul, J. 2003. ‘De mens is de maat van alle dingen’. Over mensgericht ontwerpen van producten en processen. (‘Man is the measure of all things’. On human centrered design of products and processes). Inaugural address. Erasmus Research Institute of Management, Rotterdam, The Netherlands. Dul, J. and Ceylan, C. 2006. Enhancing organizational creativity from an ergonomics perspective: The Creativity Development model. Paper presented at the 16th World Congress on Ergonomics (IEA 2006), Maastricht, The Netherlands, July 10–14. Dul, J., Ceylan, C. and Hendriks, H. 2007. A practical instrument to measure the creativity potential of the work environment. In: Proceedings of the 10th European conference on Creativity and Innovation, Copenhagen, Denmark. Dul, J. and Neumann, W. P. (in press) ‘Ergonomics Contributions to Company Strategies’, Applied Ergonomics. ELA (2004). The success factor people in distribution centers, European Logistics Association/Kurt Salmon Associates, Brussels. Koningsveld, E. A. P., Dul, J., Van Rhijn, G. W. and Vink, P. (2005). Enhancing the impact of ergonomics interventions. Ergonomics, 48(5), pp. 559–580. Lee, K. S. (2005) ‘Ergonomics in total quality management: How can we sell ergonomics to management?’ Ergonomics, 48(5), pp. 547–558. Looze, M.P. de, Van Rhijn, J.W., Deursen, J. van, Tuinzaad, G.H. and Reijneveld, C.N. (2003). A participatory and integrative approach to improve productivity and ergonomics in assembly. Production Planning & Control 14(2): 174–181. Meerding, W. J., IJzelenberg, W., Koopmanschap, M. A., Severens, J. L. and Burdorf, A. (2005). Health problems lead to considerable productivity loss at work among workers with high physical load jobs. Journal of Clinical Epidemiology, 58(5), pp. 517–523. Porter, M.E. 1985. Competitive advantage: creating and sustaining superior performance. Free Press, New York, USA. Rhijn, J.W. van, Looze, M.P. de, Tuinzaad, G.H., Groenesteijn, L., Groot, M.D. de, Vink, P. 2005. Changing from batch to flow assembly in the production of emergency lighting devices. International Journal of Production Research 43(17), 3687–3701. Sen, R. N. and Yeow, P. H. P. (2003) ‘Cost effectiveness of ergonomic redesign of electronic motherboard’, Applied Ergonomics, 34(5), pp. 453–463.
OCCUPATIONAL SAFETY: CHARTING THE FUTURE Y. Ian Noy Liberty Mutual Research Institute for Safety, USA The relentless human drive to overcome our inherent limitations and amplify our physical and mental capabilities is evident from examining the historical evolution of work systems. This paper spotlights selected milestones in the evolution of human work, illustrating the role that ergonomics innovations have had in shaping human civilization. With this as backdrop we discuss the relatively recent emergence of occupational safety as a societal priority. Yet, despite the enormous magnitude of the global burden of work-related injuries, progress has been slow. We argue that meaningful future progress in occupational safety will need to embrace the principles of resilience engineering and be integrated with modern approaches to management systems.
Introduction As the Ergonomics Society prepares to celebrate its 60th Anniversary it is fitting to reflect on the past achievements of the discipline and contemplate the prospects of ergonomics playing a more meaningful role in shaping the society of the future. The theme of the annual conference, “Contemporary Ergonomics”, challenges us to examine the current state of the discipline/profession in relation to important global issues. In this paper we examine the burden of occupational injuries as a major public health concern facing both industrialized and industrializing societies. In his book The Substance of Civilization: Materials and Human History from the Stone Age to the Age of Silicon, Stephen Sass (1999) traces how the discovery of new materials has shaped events throughout human history, enabling technological innovations and ushering in new eras. There is a hidden, yet a far deeper dimension running throughout the book that tells an even more compelling story. Read from the perspective that the discovery of new materials is but the outcome of the natural human drive to amplify our physical and mental power, what emerges is a compelling account of the evolution of ergonomics from an artform dealing with relatively simple machines to a science dealing with complex, dynamic systems. Indeed, the powerful story between the lines of this book provides a fascinating tour of the evolution of work systems. We begin with a brief description of selected milestones in the history of work systems, based loosely on Sass’ work, and with this as backdrop we discuss the emergence of occupational safety as a societal priority. From a high level perspective, workplace safety is a rather recent concern that demands far greater attention than it has received thus far. In particular, we argue 24
Occupational safety: Charting the future
25
that future progress in occupational safety will need to embrace the principles of resilience engineering and be integrated with modern approaches to management systems.
Evolution of work systems: From art to science Many associate the emergence of the science of work, with Frederick Taylor’s application of scientific principles at the turn of the 20th century to improve productivity. Actually, references to the science of work appeared 50 years earlier in Jastrebowski’s 1857 treatise entitled “An outline of ergonomics, or the science of work, based upon truths drawn from the Science of Nature”. Lexicon aside, it is evident that some form of work science was already developed thousands of years ago. It may not have benefitted from the application of the scientific method, but it was tried, tested and codified. How else could massive projects such as the pyramids of ancient Egypt be organized and carried out? How else could ingenious aqueducts that span hundreds of miles across the Roman Empire be built with great precision? Other examples of monumental projects include the Great Wall of China that extends over 6,400 km (4,000 mi) and was built in various stages starting in the 6th century BCE. These massive projects, and many more like them, required sophisticated methods to plan and organize, train workers and manage the work. Moreover, it is clear that some form of ergonomics knowledge was relatively advanced even 2500 years ago. For example, archaeological discoveries relating to ancient Greece provide abundant evidence of the extensive application of human-centred principles in the design of temples, theatres, tools and household implements, and methods relating to construction and medicine (Marmaras 1999). It is of particular interest that in 400 B.C.E., Hippocrates, the father of Western medicine, formulated a set of guidelines for surgical work in a hospital. His writings indicated that the design of the workplace, the use of tools, lighting and posture were important factors affecting the surgeon’s performance. These guidelines embodied ergonomics knowledge, though they may not have been scientifically derived.
Pre-historic work systems The fact of the matter is that the human drive to innovate and control our environment actually has its origins millions of years ago and coincides with the very foundations of society as we know it. The role and power of human-centred technology is a common thread in the evolution of human society. Early humans were by necessity constantly searching for ways to extend their capacity to survive, and ultimately to prevail. They fashioned simple tools from stone to dig up roots, crack open nuts, hunt game and to cut hides. Anthropologists have uncovered the earliest evidence of stone implements in the Olduvai Gorge in Tanzania, dating back more than two million years. This was the dawn of ergonomics as an artform. In shaping the hand ax for more complex tasks Homo habilis was applying ergonomic principles to improve biomechanical power. He had no formal
26
Y. Ian Noy
understanding of physics, biomechanics or properties of materials – but the end product was an ergonomic innovation nonetheless. It is particularly fascinating that the first appearance of stone tools coincided with the separation of the genus Homo from Australopithecus, which eventually became extinct. The aptitude for ergonomics as characterized by the intelligent and creative use of resources to support the human enterprise gave Homo a decided advantage. Homo survived for other reasons, including the sharing of responsibility for gathering wild plants and hunting game (early application of macroergonomics). Interestingly, other species, notably chimpanzees, exhibit intelligent use of tools and organized behaviour. But, humans evolved at a much faster rate. According to the renowned biologist, Christopher Wills, unlike other species that evolve in response to changes in the environment, humans evolve in response to changes that we ourselves make in our environment (Wills, 1998).
Agriculture-based society The emergence of Homo sapiens some 35,000 years ago led to further technological innovations, yet these modern humans continued to live a nomadic hunter-gatherer lifestyle for several thousands more years. The transition to a farmer-merchant economy first occurred in the Middle East some 10,000 years ago and was made possible by the discovery of clay and how to fire it. This ergonomic breakthrough facilitated the cooking and storage of liquids and grains. Moreover, by providing the means for transporting food and other products, clay vessels enabled trade with neighbouring Mesopotamia and Anatolia for needed raw materials and metals. Archaeological discoveries in Jericho suggest that its inhabitants learned to cultivate wheat and store food products. The advent of agriculture allowed them to establish permanent settlement. Two thousand years later, the technological spotlight shifted to southern Babylon. It is here five to six thousand years ago, that written language was developed for keeping inventory of farm goods, keeping records of trade, and maintaining property deeds. In the same way that stone tools extended human physical capabilities, written language extended human memory resources, and established a reliable basis for shared knowledge. Parenthetically, ancient written media also included stone. The most famous example is the Code of Hammurabi, carved in stone in 2200 B.C.E. This code is of some interest to ergonomics because it contains the first known set of design standards, and established the penalties of defective design. For example, according to the code, if a house should fall and cause the death of the owner, the builder shall be put to death. Further specialization of tools and instruments to suit the particular task at hand awaited the discovery of metals, such as copper and bronze. New tools such as saw blades and drills were developed that proved as important economically as chariots and swords were politically. Iron appeared later, being far more difficult to extract than bronze. In some regions, steelmaking was developed to improve the functional characteristics of iron. In The Substance of Civilization Stephen Sass recounts how the iron swords of the Gauls bent during their battles with the Roman legions,
Occupational safety: Charting the future
27
which were armed with Toledo steel blades (Sass, 1999). The Gauls apparently had to straighten their blades across their knee after each blow, a deadly ergonomics disadvantage. But, the Roman Empire, despite its technological prowess, eventually fell, leaving the Western world in chaos. The quality of goods and production processes actually declined during the fifth and sixth centuries. The Spread of Islam in the seventh and eighth centuries stimulated renewed interest in science and technology, and it helped to introduce Eastern ideas into Western culture, ideas that were instrumental to the continued evolution of ergonomics. Printing and paper, both developed in China about a thousand years earlier, spread westward along the Silk Road in the ninth and thirteenth centuries, respectively. The ability to reproduce hundreds of books with identical text and diagrams was crucial to the rise of modern science and technology. Another Chinese innovation to be carried westward was gunpowder. It was decisive in the Muslim resistance against the Crusaders, leading to the development of cannons and the need to improve the quality and production of iron, which was to become the engine of the Industrial Revolution. The Renaissance marked the transition from medieval to modern times. The rediscovery of Greek and Roman literature led to the emergence of the humanist movement in the 14th century, and an interest is scientific knowledge. Humanism, the belief that each individual has significance within society, was yet a further expression of the silent, but powerful, human-centred ideal. By the end of the Renaissance, another idea emerged – it was profitable to turn out products for common people. A mass market was opening, and the new economy was being fueled by private enterprises in countries that encouraged it.
Industrialization: The precursor of ergonomics as a science Together with coal, iron made possible the industrialization of the modern world. The Industrial Revolution marked the transition from a stable agriculturalcommercial society to a modern industrial society relying on complex machinery rather than tools, thus opening a new chapter in the relationship between people and technology. Whereas in England, machines were used to improve quality, in America, machines were used to improve productivity. In both instances, however, there was a need to overcome existing human-system limitations. Dramatic changes in the social and economic fabric of society took place as inventions and technological innovations created the factory system and greater economic specialization. Things began to be mass produced for unknown users – users who had different needs, were of variable size and had different preferences. The challenge then, as it is now, was to design things that would be functional and appealing to a wide market. This gave rise to an interest in individual differences and its application to design, a topic central to ergonomics science. We also note the emergence of explicit human-centred concepts in the design of machines, such as the spinning jenny. The industrial revolution also gave rise to the modern institution of labour. It led to greater intellectual interest in the work activity of ordinary people. LaMettrie’s controversial book, L’homme Machine, published in 1748, concerned the relationship
28
Y. Ian Noy
between humans and machines. By the 19th Century, labour was viewed by scientists, social and political reformers as an immense reservoir of energy that needed to be managed (conserved) so as to optimize productivity (Rabinbach, 1990). The advent of the Industrial Revolution in the mid-18th Century ushered in what we know as the science of work. The application of the scientific method to work system design was at first an attempt to maximize human output, but later reoriented to address overall system performance and worker wellbeing. The period following the Industrial Revolution was marked by an explosion of technological innovations, which continue today at an accelerated rate, leading to the recent emergence of systems engineering as an approach to address the growing complexity of socio-technical systems. More recently, the construct of joint cognitive systems underscores the fact that humans and machines must be viewed as an integrated whole whereby humans act via the machine interface to affect change rather than on the interface (Hollnagel and Woods, 2005). This draws attention to the importance of humans and machines working cooperatively to control outcome in an unpredictable environment, rather than viewing them as separate units. We posit that this rather profound shift in paradigm, from human-machine interaction to human system integration, is central to thinking about new approaches to address the issue of occupational safety.
Occupational safety: Past and present Inventive work systems that had been developed to carry out the construction of the ancient pyramids or the Great Wall of China must have involved elaborate engineering and management. However, it is clear that worker safety was not a prime consideration in the design of work. For example, it is estimated that between one and two million workers died during the construction of the Great Wall of China. Much of the labour force in ancient Egypt and the Roman Empire comprised slaves who suffered unimaginable afflictions from their work, as they did from their oppressive masters. In fact, apart from survival, the primary motivation for ergonomics innovations throughout most of human history was the drive for political or economic superiority. Social objectives were secondary, if they mattered at all. Concern over occupational health and safety emerged during the Industrial Revolution. There was already some interest in this as evidenced by Ramazzini’s book in 1717, The Diseases of Workers, linking occupational hazards to work factors including repetitive tasks, posture and stress (Ramazzini, 1717). It took over half a century for the adoption of legislation on working conditions. Perhaps the earliest Act in the UK, The Health and Morals of Apprentices Act of 1802, applied to orphan apprentices in the textile industry. It stated that no children under the age of 9 were to be apprenticed and the working day was to be limited to 12 hours with no night work. Gradually, legislation was extended to other industries and smaller establishments. To be sure, since Ramazzini’s work, considerable progress has been made in understanding the aetiology of work-related disorders and developing interventions to control the risks in the workplace. Yet, the situation today remains quite dismal.
Occupational safety: Charting the future
29
$ Billions Overexertion Fall on same level
$12.4 25.7%
Fall to lower level Bodily reaction Struck by object Struck against object Highway incident $6.4 13.3%
Caught in/compressed $5.3 10.8%
$4.8 10.0%
Repetitive motion $4.3 8.9%
Assaults/ violent acts $2.5 5.1%
$2.4 4.9%
$2.1 4.4%
$2.0 4.0%
$0.4 0.9%
Figure 1. Top ten causes and direct costs of workplace injuries in 2006. The WHO and ILO estimate that about 2.0 million work-related deaths occurred in 2000. While our knowledge of the global burden of workplace injury is woefully incomplete, it is widely acknowledged that it is enormous and unacceptable. Even in the developed world, the incidence of disabling injuries is unknown or understated. In the U.S.A., where reasonable data exist, the direct cost of the most serious occupational injuries in 2006 is estimated to be $48.6 B (Liberty Mutual Research Institute for Safety, 2008). The burden of serious1 workplace injury in the U.S.A. has been reported annually since 1998 in the Liberty Mutual Workplace Safety Index (LMWSI), which combines information from Liberty Mutual, the Federal Bureau of Labor Statistics and the National Academy of Social Insurance. These data have been influential in drawing attention to workplace injury as a public health and safety priority, and they permit prioritizing research efforts in areas that can have the most impact. Figure 1 shows the 10 leading causes of workplace injuries for 2006, as reported in the LMWSI. As shown, the leading cause of injuries is overexertion, accounting for over 25% of the burden, followed by falls on same level and falls to lower level, which together account for a further 25% of the burden. During the 8 years that the WSI has been published, the overall burden has not changed appreciably. While there have been differential changes in the relative contribution of the leading causes over the period 1998 to 2006, as depicted in Figure 2, the overall annual burden at nearly $50B remained unacceptably high. In sum, despite scientific progress in understanding the mechanisms underlying work-related injuries, efforts to develop effective countermeasures have met with limited success. We need to redouble our efforts to address this major public health issue. 1
Serious injury is defined as an injury resulting in six or more days away from work
30
Y. Ian Noy Overexertion Fall on same level Fall to lower level Bodily reaction Struck by object Struck against object Highway incident Caught in/compressed by Repetitive motion Assaults & violent acts
⫺40.0
⫺30.0
Figure 2.
⫺20.0
⫺10.0
0.0
10.0
20.0
30.0
Percent real growth change 1998–2006 by category.
Occupational safety: Where to from here? Comprehensive and meaningful improvement in occupational safety demands a change in our fundamental approach to the problem. Although considerable work remains to understand the physiological, biomechanical, behavioural and physical mechanisms underlying work-related injury, a wealth of knowledge exists to inform engineering solutions and workplace controls. It is increasingly evident that one major impediment to progress relates to the relatively underdeveloped knowledge in areas related to individual and organizational behaviour. In the remaining sections we briefly touch on topics that we believe should be vigorously pursued and developed, such as situated safety, resilience engineering and resilience management. The common theme is that safety must be regarded as an integral part of the whole work system, rather than as a separable property.
Situated safety Situated safety emphasizes the need to take into account critical situational factors such as target workforce, work organization, culture, language, infrastructure, and other socio-technical characteristics in developing an understanding of work-related injuries and prevention strategies. All too often research protocols and instruments are borrowed and applied in settings that differ in one or more fundamental ways from the settings for which they were developed or demonstrated to be valid. Similarly, interventions are implemented in situations where they have not been shown to be effective. It is essential to understand the limitations to generalizability of research protocols and interventions that appear in the literature.
Occupational safety: Charting the future
31
Related to this is the notion that safety should be viewed in relation to a dynamic situation, rather than as a probabilistic outcome associated with exposure to static risks. Humans are goal oriented and remarkably adaptive. They often work under conditions of uncertainty or subject to unforeseen disturbances. As a result, risk must be assessed in relation to the context of the operation. This requires knowledge of intrinsic and extrinsic situational factors that may be time varying and diverse, which makes them very difficult to study and measure. The case-crossover methodology (Lombardi et al., 2003) developed within the field of injury epidemiology attempts to identify the transient conditions that existed at the time of injury. Clearly, it will be important to improve our ability to capture and analyze situational data.
Safety through resilience engineering Conventional approaches to safety employ a public health model. This entails a sequence of activities designed to identify the risks, isolate the causes, study the underlying mechanisms, determine the most effective means of prevention, implement and test effectiveness. This approach is inadequate inasmuch as it is reactive and relies on hindsight. Unfortunately, there will always be new paths to failures. Rather than erecting reactive barriers and defences, the preferred approach is to design work systems that are resilient. The key to resilience engineering is building capacity to recognize, adapt to and recover from variations, surprises and failures. This is especially important for complex, dynamic systems. Hollnagel et al. (2006) summarize the current state of the science underlying this approach and provide a comprehensive description of its various aspects. They believe that “safety depends on providing workers and managers with information about changing vulnerabilities and the ability to develop new means for meeting these”. New methods and tools are beginning to emerge that can be used to engineer the system to allow workers to cope with complexity under pressure.
Safety through resilience management Business excellence models are used extensively in Europe, America, Asia and Australia to promote and reward quality management. The common element across the various models is the pursuit of continuous improvement of goods, services and processes through programs that focus on customer results and empower and motivate employees (Noy, 2000). From an ergonomics standpoint, it is noteworthy that the various models of business excellence consistently focus on people, both as customers and employees. Safety in the organization’s quality management system is viewed as an integral part of business excellence rather than an afterthought, measured at the end of the month to inform corrective measures. It is planned for and monitored on an on-going basis in real time. Resilience management recognizes the value of integrating safety and ergonomics in strategic and operational objectives. It comprises a comprehensive set of processes for anticipating, detecting, reacting and adapting to situations as they arise. The structure and processes of an enterprise should be analyzed to determine the extent to which safety considerations are systematically incorporated in
32
Y. Ian Noy
performance and quality management and constitute an integral part of business decisions.
Conclusions Resilience engineering and resilience management systems are complementary in the sense that the former relates to the design of a work system whereas the latter relates to its operation. Together they represent shifting paradigms – away from regarding safety as an isolated metric of system outcome to an integral objective of system design and operation. Work systems have evolved over millennia to a point where current systems thinking can support emerging constructs such as resilience engineering and management. Developing methods for applying these constructs represents the principal challenge for future progress in occupational safety. Until these new paradigms can be turned into practical models and tools perhaps an understanding of these concepts will stimulate designers, engineers and managers to think about occupational safety in a different, more beneficial way.
References Hollnagel, E., Woods, D.D. 2005, Joint Cognitive Systems: Foundations of Cognitive Systems Engineering, (CRC Press) Hollnagel, E., Woods, D.D. and Leveson, N. 2006, Resilience Engineering: Concepts and Precepts, (Ashgate) Liberty Mutual Research Institute for Safety. 2008, From Research to Reality, Volume 11(3) www.libertymutual.com/research Lombardi, D.A., Sorock, G.S., Hauser, R., Nasca, P.C, Eisen, E.A., Herrick, R.F., Mittleman, M.A. (2003). Temporal factors and the prevalence of transient exposures at the time of an occupational traumatic hand injury, J Occup Environ Med; 45: 832–840. Marmaras, N., Poulakakis, G. and Papakostopoulos, V. 1999, Ergonomic design in ancient Greece, Applied Ergonomics, Volume 30, Issue 4. Noy, Y.I., 2000. The Need for Integrating Human-Centred Principles in Quality Policy, keynote paper, II APERGO Conference on Ergonomics in Quality Management, Costa da Caparica. Rabinbach, A. 1990, The Human Motor: Energy, Fatigue, and the Origins of Modernity (Basic Books) Ramazinni, B., 1717 and 1940, In W. Wright (trans.): The Disease of Workers, (Chicago: University of Chicago Press) Wills, C. 1998, Children of Prometheus: The Accelerating Pace of Human Evolution, (Basic Books) Sass, S.L. 1999, The Substance of Civilization: Materials and Human History from the Stone Age to the Age of Silicon, (Arcade Publishing)
ERGONOMICS AND PUBLIC HEALTH Peter Buckle Robens Centre for Public Health, Post Graduate Medical School, Faculty of Health and Medical Sciences, University of Surrey, Guildford GU2 7TE The disciplines of public health and ergonomics share scientific concepts and often address similar health issues. However, the potential for a significantly greater engagement with public health priorities now exists. Problems to be tackled include the ageing population (and workforce), obesity and patient safety. Priorities may be set using public health methods (e.g. epidemiological trends, accident data, sickness records) but an ergonomics system approach will be required to address the complex settings in which public health must be delivered. This paper seeks to discuss how ergonomics and public health priorities might be mapped to identify suitable areas for research and knowledge application.
Introduction In this paper I have sought to set out an agenda that might identify and then strengthen the links between public health and ergonomics. It will also seek to show how new areas for research and professional application might be identified. Finally, it will identify and emphasise the importance of a number of current public health concerns. Public health has been defined as ‘the science and art of preventing disease, prolonging life and promoting health through the organised efforts and informed choices of society, organisations, public and private, communities and individuals.’ (Wanless, 2004). The breadth of this definition may be usefully set alongside the IEA (2000) definition of ergonomics ‘Ergonomics (or human factors) is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data, and methods to design in order to optimise human well-being and overall system performance.’ Both definitions stress the need to take a systems perspective. They also share the need to apply evidence based, scientific approaches when addressing systems based problems. More specifically the impact of both disciplines on protecting the health and well-being of individuals, is readily apparent. Despite these similarities there has been little consistent effort exerted by either discipline to develop potential synergies. 33
34
Peter Buckle
Public health in practice The approach to public health promoted by the USA’s Centre for Disease Control (CDC, 2008) can serve as a simple introduction. They describe a conceptual cycle, from assessment, through policy development to assurance of public health. This cycle can be segmented as follows: • • • • • • •
Monitor health status to identify community health problems Diagnose and investigate health problems and health hazards in the community Inform, educate, and empower people about health issues Mobilize community partnerships to identify and solve health problems Develop policies and plans that support individual and community health efforts Enforce laws and regulations that protect health and ensure safety Link people to needed personal health services and assure the provision of health care when otherwise unavailable • Assure a competent public health and personal health care workforce • Evaluate effectiveness, accessibility, and quality of personal and populationbased health services In addition, there is a recognition of the need to maintain active research across all of these segments to gain fresh insights to problems and to seek new solutions, against an ever changing system in which in all public health is delivered. The steps described in this public health cycle follow closely those of ergonomics practice. Similarities and natural synergies are readily identifiable. To examine more closely the relationship between public health and ergonomics we might consider topical issues. In England, the Chief Medical Officer’s annual report (2008) describes how each of the nine public health regions is prioritising health agendas. A simplified list of topics includes: • • • • • • • • • •
Mental health (and mental health and wellbeing in employment) Patient safety and acquired infections Driving down heart attack rates Uptake of screening services Health effects of climate change (e.g. heat stress) Inequality and deprivation Floods and recovery from floods (so called ‘slow burn’ emergencies) Response to emergencies (e.g. Avian flu) Childrens’ health and wellbeing Reducing childhood injuries (including falls, transport accidents, poisoning, smoke and fire, drowning) • Road deaths Each topic might be explored from an ergonomics perspective, and even simple reflection shows that most have important dimensions that can only be tackled using ergonomics and human factors based approaches. This leads logically to the challenge of how to systematically consider each from an ergonomics perspective and thus enable topics to be extracted and prioritised for research and knowledge application.
Ergonomics and public health
35
How and where might ergonomics influence public health? To address this question it seemed appropriate to start with a well established model of ergonomics and consider how public health concerns might then be mapped to it. Moray’s (2000) model was selected as one helpful perspective of the ergonomics approach. The model has been simplified to enable each level of the system to be explored (i.e. individual, team, management/organisation, legal and regulatory rules, sociopolitical). This has then be used to consider how each level of the system might be affected with any given public health challenge (e.g. obesity, ageing workforce).
Obesity In the following example I have chosen to use the example of obesity. The Department of Health have assessed what the levels of obesity in England may be in 2010 if current trends in prevalence continue. In fact, according to the Department by 2050 60% of men and 50% of women could be clinically obese. Despite this trend, it is a topic that has, to date, received little attention from ergonomists. A recent letter published in the journal Applied Ergonomics (Williams and Forde, 2008) highlights some of immediate factors of concern relating to the increase in obesity with in the UK. The authors also identifies the paucity of ergonomics research that is considering obesity in the context of work. In the following example (see table 1) I have focused on the work context of the modern office. A similar approach might have been used to consider other work contexts, for example jobs involving manual handling, sedentary professional drivers or those with a predominantly standing requirement such as often found in the retail industry or on production lines.
Ageing workforce Another major demographic change of direct impact on ergonomics is that of the ageing workforce (CEC 2003, 2004). Buckle et al. (2008a & b) are among a number of researchers attempting to identify what the issues are that will arise from the ageing workforce (largely resultant from the ‘baby boom’ years) and the change in balance of younger and older workers in workplace teams. A second and perhaps more critical factor is likely to be a shift towards more of the workforce staying in employment beyond the traditional retirement age of 65. Little is known of their ergonomics ‘metrics’ as, to date, relatively small numbers who choose to work beyond this age and those that do make this choice are usually highly selected, and a survivor population. We have yet to see what the impact will be when many more workers find it necessary to work beyond 65 because they have, for example, inadequate pension provision. Huge challenges lie ahead for ergonomists and public health professionals if cohorts of 65 plus, 70 plus and potentially 75 plus workers are to be adequately protected and helped to remain healthy, productive members of the workforce. A failure to adequately address the ergonomics issues may have
36
Peter Buckle
Table 1.
Example of an analysis of Public health and Ergonomics.
Public health demographic concern: Rise in Obesity Context: Modern office System Levels
Potential changes
Individual characteristics Physical Increased body mass and factors concomitant anthropometric changes. Probable change in physiological fitness
Examples of Ergonomic concerns
Current ergonomics standards for equipment and environments may be inadequate. Current data no longer valid for design purposes. Current equipment may be inadequate or subject to failure. Poor work organisation may encourage sedentary work throughout the whole day. This may exacerbate other physiological disorders associated with obesity. Work-rest scheduling for obese workers may need to be adjusted to allow sufficient worker recovery
Psychological factors
Stress related to public/ colleagues perception of obesity (including harassment and obesity). Impact of inappropriate work systems on sense of well-being and performance.
Motivation and well-being at work for obese workforces requires co-ordination with public health agenda, particularly in regard of education and support.
Team
Need to ‘carry’ and ‘protect’ member of team who may be more vulnerable
General understanding of teamwork and sharing of work load is poor and even less knowledge is available regarding work design and obesity issues.
Organisation
Level of knowledge, skills to assess needs
Development of appropriate needs assessment methods and training advice to be embedded in organisation. Need to develop social and business cases. Management training needs to address obesity.
Legal and regulatory rules
Existing guidance not applicable
Need to generate new data on appropriate norms and ranges. Need to validate new guidance.
Sociopolitical
Economic (benefits) pressure to keep all workers at work and maintain a healthy workforce
Informing policy makers of the benefits of ergonomics in preventing ill-health and accidents in the workplace. Promoting work system design principles
Ergonomics and public health
37
substantial implications for work, employers, and long term sickness absence and accident rates. Some other emerging public health concerns that ergonomists might choose to investigate include: • Climate change and impact on health (e.g. slow burn disasters such as flooding, human behaviour with respect to water scarcity, heat stress in design of workplaces, transport and homes, energy use and behaviour). • Public health and care of older people (e.g. design of care homes, delivery of care in community settings, patient safety, telecare design, assisted living design, slipping, tripping and falling accidents). • Transport and public health (access to public transport with an ageing population, demographic, design of transport for changed anthropometric demographic) • Nutrition and public health (need to establish sustainable behaviours, user friendly information, access to good nutrition (perhaps integrated with workplace and transport design) • Infection and public health (design of hospital systems, and other care environments, to minimize hospital acquired infections; user friendly infection control measures. • Public health and emergent threats (ergonomics in the design of bioterrorism defence systems, human factors in flu pandemics). The potential list is clearly much longer but the scope for the ergonomics discipline to be more closely engaged with the public agenda should be evident. Without the contribution of ergonomics thinking and methods the public health threats may not be tackled effectively.
Conclusions Ergonomics has, hitherto, paid little systematic and sustained attention to the public health agenda. It has much to offer, particularly from a systems methodology. Simple mapping of public health concerns to the ergonomics perspective can identify needs and help in prioritising ergonomics applications. Further, public health priorities are often based on sound epidemiological and other health trend data that provide a helpful base against which ergonomics interventions might usefully be judged. Those responsible for the education and training of professional ergonomists might consider further how opportunities for shared learning alongside public health professionals might be achieved or how a better understanding of public health concepts and priorities could be delivered. Ergonomics research and practice might seek also to influence public health through a greater sharing of research and application methods and findings. Publishing in public health journals and presenting at public health conferences are helpful options. The opportunity to showcase ergonomics within a public health context is an important one and the agenda is extensive and ever changing.
38
Peter Buckle
References Buckle P, Woods V, Oztug O, and Stubbs D 2008a Understanding workplace design for older workers: a case study In ‘Contemporary Ergonomics 2008’ Ed. Bust, P Publ. Taylor and Francis ISBN 978-0-415-46575-5 pp3–7. Buckle P, Woods V, Oztug O, and Stubbs D 2008b Understanding workplace design for older workers. Final report to Strategic Promotion of Ageing Research Capacity (SPARC) http://www.sparc.ac.uk/projects_older_worker.asp#l1. CDC 2008 The Essential Public Health Services http://www.cdc.gov/od/ocphp/ nphpsp/essentialphservices.htm accessed 10/10/08. Commission of the European Communities 2003 Sec 429 The Stockholm and Barcelona targets: Increasing employment older workers and delaying the exit from the labour market. Commission of the European Communities 2004 COM 146 final. Increasing the employment of older workers and delaying the exit from the labour market. Dept of Health 2008 On the State of Public Health: Annual Report of the Chief Medical Officer 2007. Moray N 2000 Culture, politics and ergonomics. Ergonomics, 43, 858–868. Wanless, D 2004 Securing Good Health for the Whole Population London HM Treasury. WHO 2008 Closing the gap in a generation: Health equity through action on the social determinants of health ISBN 978 924 1563703. Williams, N and Forde M 2008 Ergonomics and obesity. Letter to the editor, Appl Erg 40, 148–9.
HUMAN FACTORS AND DEVELOPMENT OF NEXT GENERATION COLLABORATIVE ENGINEERING John R. Wilson, Harshada Patel & Michael Pettitt Human Factors Research Group, University of Nottingham, UK Collaborative working, and especially that which takes place distributed in time and space, presents particular challenges to human factors. This paper draws from a large European Integrated Project, CoSpaces, which is building collaborative engineering systems in VR/AR, mobile and location aware computing. We discuss the development of descriptive models of collaboration and the use of use cases and scenario construction within user requirements analysis.
Introduction The changes we are seeing in the ways that work systems are designed and organised and in how people work have great implications for ergonomics approaches, methods, models and techniques. A notable example of this is the trend towards collaborative and distributed collaborative working, and the consequent need to determine how ergonomists will examine the strategies and performance of collaborative work teams, how they will apply these methods, and any emergent guidance for design. Advances made to help facilitate the work of distributed collaborative teams have focused on introducing new technologies into the workplace, and thus much research has concentrated on the design of computer and communication systems and their respective interfaces, but we must be careful not to neglect the design of operating procedures, job and team structures and notions related to performance such as distributed team situation awareness, mental workload and mental models. This paper concerns two interacting aspects of the human factors work on a major European project (CoSpaces) in collaborative work environments. First is the development of user requirements, feeding these through into system specifications. This is of interest for three reasons: the need to ensure that requirements embrace both system usability and the support of collaboration; the focus on a large “messy” distributed work system rather than the usual ergonomics of a single interface or product development (Wilson et al, 2003); and the dearth of literature which makes explicit how to define, construct and implement user scenarios, use cases and systems requirements in the context of collaborative systems. The second aspect is the development of a descriptive model of collaboration and collaborative working. For a notion which has been explored thoroughly within the world of computer supported cooperative work (CSCW) for many years we were surprised by the lack of a universal understanding of what collaboration comprises and therefore how to measure it. Our model has linked with project work such as: the generation of user requirements, development of evaluation frameworks and profiling tools for collaboration readiness. 39
40
J.R. Wilson, H. Patel & M. Pettitt
This paper draws together the work carried out on the collaboration model and that on user requirements, within the context of CoSpaces. Difficulties faced of general interest included integrating the work of human factors with the software engineering teams and disparate views regarding models within this context. There are differing views on the degree of formality required in such models and the extent to which they should be descriptive rather than predictive. This is of course an old debate within and around human factors, for instance found within discussions of mental models, and shows no signs of going away.
CoSpaces and relevant human factors We have carried out work over years in a variety of settings and for a variety of purposes, all with the theme of distributed working. This distribution is across functional specialisms amongst co-located work groups – for instance in control rooms (ATC, rail etc.) and manufacturing shop floors (electronics, automotive), and is also temporal and spatial, within mobile or distributed workspaces (e.g. in international aerospace manufacturing). This paper is based primarily on current work carried out in a large European Commission funded Integrated Project, CoSpaces, in which there are three workspaces – co-located, distributed and mobile. CoSpaces has industrial, research and business partners from 12 European countries, developing innovative collaborative working solutions that are responsive to industrial needs. The project explores how advanced technologies (virtual, augmented, tele-immersive, mobile, and context aware) can be deployed to create collaborative workspaces for supporting product design, assembly/construction and maintenance activities in the aerospace, automotive and construction sectors. Early on we structured the involvement of the human factors specialists into the overall project in such a way as to be as supportive as possible of the technical development teams, acting as the representatives of the users and translators of their requirements. Another role was to ensure that the needs for collaboration (and usability) were not swamped in the rush to produce advanced architectures and exciting tools and those resultant systems could be implemented to benefit engineering and business performance. Figure 1 illustrates the human factors contribution – of course a too neat and optimistic process but something against which to assess the project.
Model of collaborative work Approaches to modeling One of the critical early tasks in CoSpaces was to develop collaborative models to inform the development and implementation of CoSpaces systems. However, clarity was needed on what we intended by models in this context, since the term is used quite validly in many different ways by different people. This is perhaps not so surprising when we consider that a model is merely a simplified representation of the relevant properties of an entity produced for a specific (often explanatory) purpose.
Next generation collaborative engineering: Human factors and development
41
Active distributed development space Design guidance
Visionary scenarios Partner involvement Collaboration model
Current practice scenarios
CoScope profiling
CoSpaces scenarios
Use cases
Requirements Functional specification
Evaluation framework
Development
External company involvement Evaluation
Figure 1.
CoSpaces route to functional specification and development.
The human factors team wanted a model for explanation – to give a better understanding of what is collaboration and collaborative working, and the attributes which influence and form part of collaborative work. Such a model would allow recognition of the degree and type of collaborative work currently taking place within the user partners, help to establish strengths and weaknesses, provide a framework within which to define user requirements, and subsequently inform change management strategies and evaluation. On the other hand the developers and technologists (but interestingly not the user partners) wanted to initiate development of models that could be implemented within CoSpaces systems development, primarily to be used computationally, however crudely, at the very least to allow some form of built-in user profiling and possibly to support development of intelligent agency within the workspaces. These formal models are only likely to represent simple and predictable processes, and unable to model dynamically changing, complex, and unpredictable organisational processes (Robin et al, 2007). We have concentrated upon a descriptive model, which in any case would be required to form the basis of more formal models.
Collaboration and collaborative working models Research into collaboration spans a number of disparate fields like organisational/social psychology, CSCW, management science, engineering and human factors, and domains including education, healthcare, etc.; thus existing definitions of collaboration are numerous, varied and often domain specific. However a number of common themes emerge, i.e. collaboration involves two or more people engaged in interaction, and that these people are working together towards common goals. Other associated attributes which emerge are: trust, communication, coordinating activities, shared resources, shared rules/norms, shared vision and responsibility, and shared skills. These attributes have been incorporated into our model of collaborative work. Collaboration tasks, processes, and related technology support have been classified according to various theories and frameworks, including: the group task
42
J.R. Wilson, H. Patel & M. Pettitt
circumplex, task technology fit theory, media richness theory and coordination theory (Weiseth et al, 2006). Such classification schemes are an important focus for CSCW research, and benefit from an understanding of collaboration. Many models have been proposed seeking to structure the factors influencing collaboration (Girard and Robin, 2006; Hawryszkeiwycz, 2006; Harvey and Koubek, 2000; Warner et al, 2003 and Weiseth et al, 2006), Of the few existing collaboration models there is a tendency towards simplicity or to focus on only a small part of collaborative work. Other researchers have focused on developing models to aid evaluation of collaborative work, e.g. Neale et al (2004), who presented a model for evaluating activity awareness, in which they defined the underlying processes of collaboration as being communication, coordination, and work coupling. Some researchers have attempted to isolate the core actions and interactions that form part of collaborative work. Pinelle et al (2003) suggest that these ‘mechanics of collaboration’ are independent of organisational culture, individual differences, or the type of task being conducted, and include: communication with team members, gathering information, negotiating access to shared tools and transferring artefacts to others. However, this approach does not explicitly consider high level collaboration activities such as planning, decision making and exploration. A number of researchers have attempted to model different aspects of team collaboration (e.g. Rogers and Ellis, 1994, McNeese et al, 2000). There appear to be few empirical studies which have examined collaborative work and those that have been reported have tended to use a questionnaire and/or interview approach (exceptions are the work of Vande Moere et al, 2008; Détienne et al, 2006). At the root of problems associated with evaluating collaborative systems is that we do not have a universally agreed understanding of what it means to collaborate, and thus what enhances or detracts from effective collaborative work. Such understanding is needed to determine methods to measure and predict the outcomes of collaborative work and make recommendations on enhancement and design.
The CoSpaces collaboration model We have isolated the main factors forming and influencing collaborative work through a literature review across many domains, embedded work with industrial organisations, interviews, workshops and expert brainstorming sessions, experimental studies and our own user requirements elicitation work with CoSpaces partners. The factors forming our current working representation of collaborative work are summarised in Table 1. As with many models, representation is critical. Our first attempt used three main building blocks of collaboration: collaboration environment, collaboration process and collaboration support (see Figure 2, left). However it was felt this representation implicitly suggested a process flow and an arbitrary hierarchy. We could not be confident of the implied ‘weightings’, nor of any causative influences or other factor relationships implied by the structure. The second representation shows factors in embedded elliptical layers (Figure 2, centre), illustrating the close relationship between the factors of collaboration: actors, groups, process, tasks, support and context. In contrast to the first
Next generation collaborative engineering: Human factors and development
Table 1.
43
Factors (main and sub-factors) of collaboration and collaborative work.
Individuals
Teams
Interaction Processes
Skills
Roles
Communication Type
Wellbeing Composition Psychological Common ground factors Shared awareness and knowledge Relationships Group processes
Coordination Decision making Learning Error management
Tasks
Support
Context
Knowledge Culture management Structure Teambuilding Environment Demands Training Business climate Resources Organisational structure Networks
Tools
Figure 2. Different representations of the CoSpaces Collaboration Model (note: figure to show representations, not detailed content).
version, this format does not suggest specific weights, representing all factors as interdependent and part of an overall context. We include more sub-factors in this second version. The third version is presented as a web of factors (Figure 2, right), illustrating the mutually dependent relationship between the factors. Individuals and teams are central to the process of collaboration, engaged in intraand inter-group collaboration. They are involved in interaction processes which are required to collaborate on performing tasks, which in CoSpaces form part of their design and engineering work. Providing support is essential for ensuring that collaborative work is effective and efficient, and that individuals and groups have access to the resources required to perform their tasks, and meet their goals and needs. Context forms the final segment of the web, usually dictating the actors and groups, tasks, support that is needed/provided, and will impact on the actual process of collaboration itself. Sub-factors are associated with these main factors within the web, and factors which ‘overarch’ the main factors (relevant across individuals, teams, process, tasks, support and context) are shown outside the web. We are currently exploring relationships between selected factors in laboratory and field settings, for instance testing any relationship between trust, culture and
44
J.R. Wilson, H. Patel & M. Pettitt
performance, or between familiarity, communications, shared awareness, common ground and performance.
Requirements, scenarios, use cases and functional specifications CoSpaces presents several challenges for a requirements elicitation process. First, the project is focused on three industrial sectors, with four user partners, who are themselves groupings of a number of smaller partners. This is in stark contrast to the more normal development process that considers either a bespoke system for a single organisation or user/user group, or a general application for a wider consumer market. Second, the project intends to develop a system that supports collaborative working in three contexts: co-located, distributed and mobile. Within each user partner different combinations or instances of these collaborative workspaces may or may not be required. Third, the project has not started from a tabla rasa but has grown from a previous ‘road-mapping’ European project, FutureWorkspaces. Some of the CoSpaces partners (user and developer) were involved in this project, and therefore bring a certain amount of fore-knowledge with them. Others are newcomers and consequently the level of familiarity regarding the vision of the project, the ways in which members work together, and with the technologies being developed, varies between project partners. Fourth – particularly for the consortium newcomers – the nature of the collaborative technologies involved, and the underpinning virtual and distributive technologies within them, mean that potential uses, and more importantly potential work organisation and work system structures that might result from their use, can be unclear. As a result of these special challenges the user requirements gathering: was based upon an identification and prioritisation process which is structured, clear and traceable; was as far as possible user-centred and beyond this user-partnered; resulted in prioritised, verifiable requirements, which can be translated into practical specifications. The seven stage process is: Stage 1 – Raising enthusiasm and creativity through the use of visionary scenarios, stories, and technology demonstrations. CoSpaces began by demonstrating the potential vision for collaborative workspaces to establish as clearly as possible a common starting point. Technical setups were presented allowing user partners to identify commonalities and potential uses within their organisational settings. With such advanced virtual and distributed technology, traditional user requirements identification may not work; users’ needs should be neither too “Star Wars” (unrealistic visions) nor too earth-bound in today’s technology. Ambient interfaces, context-aware functionality etc. may be poorly understood by end users, who are unlikely to be aware of tangible benefits, likely frequency of use, nor potential problems. Conceptual scenarios were potentially powerful tools for later requirements elicitation but, early on, demonstrations of related systems were useful for stimulating discussion, enabling users to more clearly define what they want or could possibly use and to make more informed decisions, leading to better communication and collaboration between the users and the designers/developers (Carroll, 1997).
Next generation collaborative engineering: Human factors and development
45
Stage 2 – First involvement and visits to user companies, in order to foster relations with user partners, gain an initial understanding of the collaborative contexts to be further investigated, and establish a full list of stakeholders. Stage 3 – Development of current practice scenarios for collaborative engineering. Beynon et al (2005) have differentiated between: user stories, defined as real world experiences, ideas, anecdotes and knowledge, activity and context produced in any medium; conceptual scenarios, which report common elements across stories and are perhaps equivalent to the essential use cases of Constantine and Lockwood (1999); concrete scenarios, of which several may be drawn up for one project; and, use cases (see Stage 5). The value of scenarios, for CoSpaces is that they provided a means to an end to elicit user requirements in a form subsequently usable for system specifications, and system development. Scenarios allowed user requirements analysts to explore the context, user needs, and requirements in a vocabulary acceptable to stakeholders, whilst also communicating the priorities of those stakeholders to developers. We created rich-detail, tabulated descriptions of current practices, focussing on task and process level detail, organisational goals, stakeholder information, key decision stages, the types of communication that occur and the organisational, social and cultural issues surrounding collaborative behaviour, as a gateway developing user requirements within context and opening a dialogue between researchers and users for discussion of the potential applications. Stage 4 – Detailed development of current practice scenarios through semistructured interviews. Building on work in the previous stage, and conducted with real end users, human factors researchers were able to follow up in more detail the information gathered in the scenarios. Interviews enabled: development of all scenarios to comparable levels of detail; discussion of where technology might be used to improve collaborative practice; and the opportunity to observe current practice first hand, focussing particularly on the key activities, communications, collaborative relationships and decision making processes involved in the described task(s). Researchers were able to identify problems that could be solved by introducing new technology or through organisational changes. In accordance with the fundamental philosophy of socio-technical system design, both types of need emerged from the interview process. For example, a common theme was that, whilst current tools used were not especially advanced – requiring significant set-up costs and not dealing well with large data files, and without the functionality to share CAD models – these were not in themselves enough to explain poor collaboration performance. A combination of technology solutions and organisational approaches was required, e.g. policies and incentives to encourage the use of collaborative tools. Stage 5 – Generation and agreement of use cases. Researchers and users worked together to produce specific use cases for new technology within the scenario settings (e.g. Cockburn, 2001). Use cases describe the interaction with a system yet to be designed in terms that are easily understood, concentrating on what people try to achieve, rather than how they achieve it. In CoSpaces use cases were textual, in structured stages. Stage 6 – Detailed user requirements production and prioritisation. Requirements were elicited for each step through a use case, rooting them firmly within the
46
J.R. Wilson, H. Patel & M. Pettitt
use case and scenario environment (Maiden and Mavin, 2006). The aim was to articulate as requirement statements the precise functionality required to support the behaviour and activity described in each case. Requirements were then compiled into one document, synthesised, combined and categorised; by researchers working with user partners, and prioritised according to importance (high/low) and timescale (short-term/long-term) by both development and user partners. Stage 7 (and beyond) – System specification production and negotiation. The final stage involved discussion primarily between research and development partners, in order to clarify the meaning and interpretation of requirements, and determine technical feasibility. This process allowed developers to better understand the underlying needs behind the requirements and suggest initial groupings and technical solutions.
Next steps and conclusions Naturally, a good user-centred design process does not begin and end with requirements elicitation. Particularly for collaborative systems, where the ‘task’ of collaboration is often vague, transitory and difficult to specify, we need to ensure close contact with users throughout the technical development into real systems. In sympathy with this, CoSpaces is following a Living Labs approach, whereby we aim to test and develop throughout the project, involving users as much as feasible, and carrying out iterative development at user sites where practical. The Living Labs notion is not without its critics, but our approach is twofold: development of a digital community infrastructure, especially integrating CWE development across small suppliers in the motor industry; and methodological, akin to the “digitised observation home”. We have termed the second an Active Distributed Development Space (ADDS), through it bringing research into reality rather than reality into research and using the notion of design partners within a tradition of participatory and user-centred design (see Wilson et al, 2008). We are currently exploring how the collaboration model could be used to measure (quantitatively or qualitatively) the collaboration which takes place in a company by applying weightings to each factor in the model to represent how people currently work and how they would like to work in the future. This work has the ultimate intention of helping industrial organisations improve their collaborative performance across tangible (e.g. improved product development efficiency) and implicit (e.g. greater employee satisfaction) dimensions.
Acknowledgements This paper is based on work carried out within the EU Integrated Project CoSpaces.
References Beynon, D. Turner, P. and Turner, S. 2005, Designing Interactive Systems (Pearson Education, Essex)
Next generation collaborative engineering: Human factors and development
47
Carroll, J.M. 1997, Scenario-Based Design. In: M. Helander, T.K. Landauer, P. Prabhu (eds.) Handbook of Human-Computer Interaction (Elsevier, Amsterdam) Cockburn, A. 2001, Writing Effective Use Cases (Addison-Wesley, London) Constantine, L.L. and Lockwood, L.A.D. 1999, Software for Use (Addison-Wesley, Harlow) Détienne, F., Burkhardt, J.-M. & Barcellini, F. 2006, Methodological principles to analyse distant and asynchronous collaborative design, Symposium on Methodological principles for analysing and assessing collaborative design, International Ergonomics Association Conference, Maastricht, The Netherlands, 10 to 14 of July 2006 Girard, P., and Robin, V. 2006, Analysis of collaboration for project design management, Computers in Industry, 57, 817–826 Harvey, C.M., and Koubek, R.J. 2000, Cognitive, Social and Environmental Attributes of distributed Engineering Collaboration: A Review and Proposed Model of Collaboration, Human Factors and Ergonomics in Manufacturing, 10(4), 369–393 Hawryszkiewycz, I. 2006, A framework for raising collaboration levels on the internet. Lecture notes in Computer Science, 4082/2006 Maiden, N. and Mavin, A. 2006, WHERE: Scenario Modelling, In The IET Seminar on: Answering the Six Questions About Requirements (The IET, London) McNeese, M.D., Rentsch, J.R., and Perusich, K. 2000, Modeling, measuring and mediating teamwork: The use of fuzzy cognitive maps and team member schema. Proceedings IEEE International Conference on Systems, Man and Cybernetics, (Institute of Electrical and Electronic Engineers, New York), 1081–1086 Neale, D.C., Carroll, J.M., and Rosson, M.B. 2004, Evaluating computersupported cooperative work: models and frameworks. Proceedings of CSCW ’04, November 6–10, 2004, (ACM Press, USA), 112–121 Pinelle, D., Gutwin, C., and Greenberg, S. 2003, Task analysis for groupware usability evaluation: modelling shared-workspace tasks with the mechanics of collaboration, ACM Transactions on Computer-Human Interaction, 10(4), 281–311 Robin, V., Rose, B., and Girard, P. 2007, Modelling collaborative knowledge to support engineering design project manager, Computers in Industry, 58(2), 188–198 Rogers, Y. and Ellis, J. 1994, Distributed cognition: an alternative framework for analyzing and explaining collaborative working, Journal of Information Technology, 9(2), 119–128 Vande Moere A., Dong A. and Clayden J. 2008, Visualising the Social Dynamics of Team Collaboration. International Journal of CoCreation in Design and the Arts (CoDesign), Taylor & Francis, pp. 151–171. Warner, N.W., Letsky, M., and Cowen, M. 2003, Structural model of team collaboration, Office of Naval Research, Human Systems Department, Arlington, VA, unpublished manuscript, http://www.au.af.mil/au/awc/awcgate/navy/model_of_ team_collab.doc
48
J.R. Wilson, H. Patel & M. Pettitt
Weiseth, P.E., Munkvold, B.E., Tvedte, B. and Larsen, S. 2006, The wheel of collaboration tools: A typology for analysis within a holistic framework, Proceedings of the 2006 Conference on Computer Supported Cooperative Work, Banff, Canada, (ACM Press, USA), 239–248 Wilson, J.R., Jackson, S. and Nichols, S. 2003, Cognitive work investigation and design in practice: the influence of social context and social work artefacts. In: E. Hollnagel (ed) Cognitive Task Design, (CRC Press, USA) 83–98 Wilson, J., Patel, H., Pettitt, M. and Saikayasit, R. 2008, ADDS: Living Labs for collaborative engineering, in J. Schumacher and V.P. Nitamo (eds.) European Living Labs: a new approach for human centric regional innovation, (Wissenchaftlicher Verlag, Berlin)
ACCESSIBILITY
A DESIGN ERGONOMICS APPROACH TO ACCESSIBILITY AND USER NEEDS IN TRANSPORT R. Marshall1 , D.E. Gyi3 , K. Case2 , J.M. Porter1 , R.E. Sims1 , S.J. Summerskill1 & P.M. Davis1 1
Department of Design and Technology, Department of Mechanical and Manufacturing Engineering, 3 Department of Human Sciences, Loughborough University, Loughborough, UK
2
This paper describes research carried out into the area of accessibility and ‘design for all’. The Accessibility and User Needs in Transport (AUNT-SUE) project was initiated to develop and test sustainable policies and practice that would deliver effective socially inclusive design and operation in transport and the public realm. Loughborough University’s role in the project focuses on the provision of data on users that is accessible, valid, and applicable and a means of utilising the data to assess the accessibility of designs during the early stages of development. These needs have led to the development of our inclusive design tool called HADRIAN. Data were collected on 100 people the majority of whom are older or have some form of impairment. These data include size, shape, capability, preferences and experiences with a range of daily activities and transport related tasks. These are partnered with a simple task analysis system. The system supports the construction of a task linked to a CAD model of a design to be evaluated. The task is then carried out by the virtual individuals in the database. Accessibility issues are reported by the system allowing excluded people to be investigated. Thus HADRIAN supports designers and ergonomists in attempting to ‘design for all’ by fostering empathy with the intended users, meeting their data needs through an accessible and applicable database and providing a means of gaining some of the feedback possible with a real user trial at a much earlier stage in the design process.
Introduction Research has identified that there is a clear and well established need for all those involved in the design of products, services and environments to take an inclusive approach, and avoid discriminating or disadvantaging users based on their size, shape, age, abilities, needs, or aspirations (Coleman et al., 2003). This need is driven by a number of factors: the ageing population (WHO, 2008), increasing legislation 51
52
R. Marshall et al.
(DDA, 2005), and a strong moral case that design should not be embodying a philosophy that prevents certain people from using, and enjoying the outcomes that it produces. The persuasiveness of these drivers together with the perceived support for inclusive design amongst the design community would suggest that evidence of inclusively designed outcomes would be common place. However this is often not the case with many designs still primarily catering for the younger and able-bodied. We believe that there is, therefore, a need for a new approach in order to effectively support designers when attempting to ‘design for all’ that must take into account a number of important issues: • Tools are often data driven. If the data is unavailable, inaccurate, misleading, unapplicable or just difficult to apply then often the tool is of limited value. • Even with accurate, applicable data, tools provide an additional layer of activity and often require some expertise. If a design team lacks the expertise in a particular discipline tools may be used to mitigate against this. Assuming tools are easy to use it can often be the case that the tool is used without a full understanding of the results or advice provided by the tool. • Tools are rarely good at addressing the softer, cognitive or emotional issues within design. Even if the tool manages this element of design well, it is not straightforward what needs to be done. This paper describes our current research as part of the AUNT-SUE project to provide better support for designers in their efforts to design for all. AUNT-SUE (Accessibility and User Needs in Transport for Sustainable Urban Environments, Porter et al., 2006) is a consortium of UK academic institutions including London Metropolitan University, University College London and Loughborough University, together with local councils and other public and private bodies such as Camden Council, Hertfordshire Council, and the RNIB. The consortium’s aim is to produce methodologies for sustainable policies and practices that will deliver effective socially inclusive design and operation of transport and is funded as part of the EPSRC’s SUE programme.
Our approach to inclusive design Our current approach to inclusive design was driven by an early survey of 50 designers which aimed to identify the current situation in designing when taking into account the needs of older and disabled people (Gyi et al., 2000). The results highlighted that available data tends to be ‘patchy’ and rarely in sufficient detail to enable design professionals to make more informed decisions. In addition, existing data tools are not in a format or language that designers can access and relate to easily. Finally it was noted that the majority of designers used at least one computer aided design package. These findings have also been identified amongst a broader range of concerns for businesses in a more recent study by Goodman et al. (2006). From this we identified that there is a clear need to provide ergonomics data in a highly visual form and to integrate design for all philosophy into existing good
A design ergonomics approach to accessibility and user needs in transport
53
practice, such as the use of CAD and other computer based design tools such as SAMMIE (System for Aiding Man Machine Interaction Evaluation, Porter et al., 2004). Human modelling systems such as SAMMIE allow ergonomics issues to be explored in a CAD environment during concept design. The benefits of being able to explore issues such as fit, posture, reach and vision at an early stage of the design are invaluable in achieving products that fully accommodate users. There are many issues when trying to apply ergonomics data to design and the same is true of ergonomics tools such as human modelling systems. Sources of anthropometric data often have limitations on how representative they are for any given design exercise. Concerns with the age of the data, access to data on appropriate Nationalities, the support for designing from 5th–95th %iles, the lack of task specificity, and so on. These concerns are compounded when the vast majority of the existing data do not relate to older or disabled people. In addition, the successful use of such tools is often coloured or constrained by the need for ‘expert’ users. For ergonomists to truly support widespread design practice, we need to develop and communicate information and methods that meet the needs of designers themselves. In response to these issues it was decided to develop an inclusive design tool. This tool is called HADRIAN (Human Anthropometric Data Requirements Investigation and Analysis, Marshall et al., 2004) and works together with SAMMIE to target the first two bullet points outlined in the introduction, namely: the provision of relevant, accessible and holistic information on people of a broad range of size, shape, and ability and a means of utilizing the available information to assess the inclusiveness of a proposed design. Together the tools support ‘playing’ with ideas at an early stage in the design process and provide feedback as to the impact of those ideas on users. In principle, the tools take the model of fitting trials, or user trials that are performed with real people and real products and move that into a virtual world where the costs and complexities of the real world are avoided and yet ‘some’ of the valuable feedback is available in a much more timely manner.
Design relevant data on people HADRIAN comprises a database of physical and behavioural data on 102 people covering a broad range of ages and abilities. The age of people in the database ranges from 18 to 89 years with 46 people who are over 60. Within the sample 59 people have some form of impairment including: limb loss, asthma, blood conditions, cerebral palsy, epilepsy, head injuries, non specific knee problems, multiple sclerosis, arthritis, vision and hearing impairments, heart problems, paraplegia, Parkinson’s disease, stroke, and dyslexia, amongst others. Of the 43 able bodied people 20 were aged over 60 and had undiagnosed or minor impairments associated with being older. In addition, the sample also contains people who reported difficulties with using some forms of transport impaired by issues such as pushchair use. The sample is clearly not representative of the more general population. To address the lack of data in this area and to directly support designers in inclusive
54
R. Marshall et al.
Figure 1.
Reach envelopes define the boundary of reach capability for an individual.
design it was a deliberate decision to skew the data towards those who are older and/or with some form of impairment. Whilst the sample is biased it has been carefully selected to cover as broad a range as possible for every measure recorded. The data consist of 26 Anthropometric body measures and 18 joint angle ranges of motion (see Table 1 for a summary of all the available data). For those people who use a wheelchair four wheelchair measures were also taken. A somatotype value is used to record body shape (Carter and Heath, 1990) in addition to two digital photographs taken from the front and side of the subject. Reach range is recorded in the form of an envelope of coordinate data centred upon the shoulder. These coordinates describe a surface mesh representing the boundary of reach capability for the subject’s arm (Figure 1). In addition to traditional anthropometric measures the data also contain a whole body scan. Using a [TC]2 whole body scanning system, subjects have been scanned to capture their body form. Body scan data is not available for all of the people in the database due to a number of limitations with the system. In particular, wheelchair users could not be scanned in their own wheelchair as the chair would interfere with the scanning process. In order to address issues with the applicability of existing data sources it was decided that the database should contain functional data in addition to anthropometry. Based upon a survey of user needs, conducted with 50 older and disabled people (Oliver et al., 2001) three areas were identified for investigation: a range of generic kitchen based tasks, a range of seating scenarios, and ingress and egress from a range of public transport types. Taking a pragmatic approach the data collection focused on tasks that were sufficiently specific to be relevant to design needs, yet generically applicable so that we were not designing a kitchen design tool, or creating a system that required data on every possible task situation in order to be useful. Using a kitchen rig consisting of wall and floor units together with a mock up of an oven, participants were asked to pick and place a range of one handed and two handed loads to various locations. A second rig was used to simulate entering and exiting from UK rail, coach and bus vehicles with a representative range of step heights and
1. Anthropometry (mm) Stature Weight Arm length Upper arm length Elbow-to-shoulder (link) Wrist-to-elbow (link) Abdominal depth (standing) Abdominal depth (sitting) Thigh depth (standing) Thigh depth (sitting) Knee-to-hip (link) Ankle-to-knee (link) Ankle height Foot length Sitting height Sitting shoulder height Hip-to-shoulder (link) Chest height Chest depth Head height Eye-to-top-of-head Buttock-knee length Knee height Shoulder breadth Hip breadth Hand length Hand grip length Wheelchair length Wheelchair height Wheelchair width Wheelchair seat height 6. Task capability (encoded postures for each task plus task videos) 4 pick & place tasks (high shelf, worksurface, oven, low shelf) with 3 load types (cup, bag, tray) each set to maximum comfortable weight, 1 or 2 hands as appropriate. Seating: 2 designs – high & hard, low & soft; restricted access to single side (bus), both sides (toilet cubicle), no restriction.
5. Whole body scan (VRML file)
4. Somatotype (3 digit number)
3. Reach range (∼100 coordinates mm) Functional reach volume generated by dominant arm/hand
9. Background Age Nationality Occupation / work history Handedness Disability Front and side photographs
8. Transport questionnaire (question and answer transcripts and videos) Transport use (frequency etc) Issues with transport usage (problems, assistance required etc) Issues with lifts, steps, escalators Issues with environment (personal safety etc) Issues with signage and timetables Local issues
7. Additional capability Bending to touch toes Getting up from lying down Reaching to tie shoelaces Twisting upper body to left and right Peg test (dexterity) Grip strength Vision
Ingress / egress: step up / step down from maximum comfortable step height, two handle types, maximum of 4 handle locations
Summary of data in HADRIAN database.
2. Joint constraints (deg) Shoulder extension/flexion Shoulder abduction/adduction Upper arm extension/flexion Upper arm abduction/adduction Upper arm medial/lateral rotation Elbow extension/flexion Elbow pronation/supination Wrist extension/flexion Wrist abduction/adduction
Table 1.
A design ergonomics approach to accessibility and user needs in transport 55
56
R. Marshall et al.
Figure 2.
Data in the database is structured around individual people.
handle locations. Where possible the data collected reflects real-world application. Thus, comfort maximums were recorded to reflect what the subject would be likely to attempt in reality where absolute maximums would not normally be used. All task activity was recorded from two positions via digital video camera and subsequently used to provide video clips of tasks being performed. Task data stored within the database includes a success or a failure for each task element. In addition, the data also records how a task was performed in terms of coded task behaviour. AUNT-SUE has also seen an expansion of the database beyond the physical into cognitive, emotional and sensory data associated with travel. Whilst these are often complex they are also the source of some of the most fundamental issues in transport accessibility. Using a questionnaire on transport activities, data were captured for each individual’s ability to deal with tasks such as using trains, buses, trams, London-style taxi cabs and minicab taxis; as well as issues surrounding travelling with luggage; the types and frequency of journeys made; problems using stairs, lifts or escalators; route planning, dealing with crowds, understanding signs and other public information, and perceptions of crime and personal safety. Thus the questionnaire provides information concerning issues that may arise at any point during the whole journey process. A key feature of the database is how the data are presented (Figure 2). To address concerns with designers being presented with faceless numbers, with the lack of visualization in current tools, and in the complexities of multivariate accommodation, the database is effectively a catalogue of individuals. Data are not broken down into categories of individual measures but are instead maintained as a set associated with a single person. Thus the user can browse through the people in the database see a picture of that individual and explore the data about that person. This approach fosters empathy between the designer and the people who they are
A design ergonomics approach to accessibility and user needs in transport
57
designing for, and attempts to minimize the dehumanizing effects of percentiles and of the virtual environment in which the design is being created.
A means of applying the data to design problems To address the utilization of the data to assess the inclusiveness of a proposed design, HADRIAN also includes a simple task analysis system. HADRIAN focuses on an integrated approach to support the designer in defining a task description associated with using their design, subsequent analysis of the task performed with the design, and final result reporting and analysis feedback. This approach then aids the designer in the evaluation of a specific design, establishing a form of semiautomated virtual fitting trial. This fitting trial can then be performed many times using the individuals in the database as virtual subjects. The analysis system was designed to facilitate a means of describing how a product would be used. ‘Product’is taken in its broadest context and may include any object that requires physical interaction. The system allows a complete task such as “purchase ticket from ticket machine” to be broken down into recognizable elements such as “select ticket type” or rather its interpretation as “reach to touch-screen”. The dynamic process of performing a task is broken down into static ‘frames’ associated with these elements. SAMMIE is then used to model the elements of these static frames including a posture for the human model, a target object, and an environment. The system then responds to each individual task element in a chain, performing physical interactions, and assessing the success of each element towards completing the task. The actual process of describing a task element involves selecting a task command such as Look, Reach, View, Move and so on. The command then needs a target that will be represented within the model to be analysed. This target will be the focus of the task element and form a verb + noun pairing such as Look at Screen, Reach to Card Slot. The task definition is then completed through the specification of task parameters. This may include an acceptable viewing distance, a particular grip type to adopt, or what limb a reach should be performed with. In an attempt to make the analysis as realistic as possible the system encodes the task behaviour recorded in the database. This behaviour is then replicated when an individual in the database is asked to perform a task of a similar nature. Thus, HADRIAN synthesizes how the task is performed by each individual in the database as the task progresses. Whilst a full treatment of the analysis model is beyond the scope of this paper, various factors are taken into consideration during this process including the dominant hand of the individual, the current orientation of the individual, the nearest limb to the target, any impairments to the non dominant limb, the details of the next task element, and so on. Irrespective of the underlying complexity of the process, the designer sees individuals in the database attempting to perform the task as defined. Each individual will have their own size, shape, capability and behaviour represented in the human model. The individuals will all perform the task slightly differently based on the
58
R. Marshall et al.
process outlined above. The outcome of the analysis is then reported to the designer in terms of a percentage excluded. The percentage is based upon those individuals in the database and so if 5 of the 100 people fail to achieve the task then 5% are reported excluded. The designer can then explore those who failed, identify exactly what individuals experienced problems, with what part of the task and why. The designer may now explore the affects of changes to the design or even the task. Because the process is occurring during an early stage of the design, possibilities can be rapidly explored and an optimum quickly identified.
Ongoing developments HADRIAN’s development has been targeted at addressing discrete design problems or products. However, the concept of inclusive transport is not solely related to any single piece of design, rather it concerns a network or system of designed elements. These designs could include everything from a flight of steps to a train carriage and yet they all form potential barriers to travel. This network is part of the transport infrastructure, combining a number of directly related, and indirectly related design problems that must be addressed holistically if inclusive transport is taken in the context of the ‘journey’. To succeed in providing truly accessible transport we must be able to ensure that a door-to-door journey for example, from home to the doctor, or from the bank to the theatre is possible at every stage. Two tools are being developed to address this need; the Inclusive Journey Planner and the Journey Stress Calculator. The inclusive Journey Planner addresses the needs of end users by providing improved information on accessibility along the whole length of a journey. The system then allows much more informed decision making on route and transport mode choice before the journey is undertaken. The Journey Stress Calculator supports the needs of transport professionals by allowing HADRIAN style analyses of a whole journey. This enables transport professional and practitioners to identify which people are most likely to avoid a journey, compare the accessibility of different routes and ensure spending is targeted at removing stressors that cause greatest exclusion (Davis et al., 2009). HADRIAN is also going through a process of validation (Summerskill et al., 2009). Validation will consist of three phases that will compare the findings from: HADRIAN used by a designer, SAMMIE used by an experienced ergonomist, and a real user trial conducted by an experienced ergonomist. All three assessments will use people from the HADRIAN database in real or virtual form to assess a number of existing transport related designs. The results will be used to refine HADRIAN.
Conclusion HADRIAN has been developed to support designers and ergonomists in developing inclusive products addressing limitations with current data and tools. The
A design ergonomics approach to accessibility and user needs in transport
59
HADRIAN database provides extensive data on 102 individuals including their size, shape, ability, behaviour, and range of other information on their background and specific transport related experiences. This data can then be employed through a simple task analysis system to assess the inclusiveness of a design. This predictive process greatly increases the chance that designers will identify issues that users may experience with a design long before the design is complete. HADRIAN is currently undergoing validation to assess its performance and to ensure that it is useable by the people it was designed to support. The database and analysis system will be made available to designers to support their efforts and help establish inclusive design as an integrated part of design practice.
References Carter, J.E. and Heath, B.H., 1990. Somatotyping Development and Applications, Cambridge: Cambridge University Press. Coleman, R., Lebbon, C., Clarkson, J. and Keates S., 2003. From Margins to Mainstream, In: J. Clarkson et al. eds, Inclusive Design, design for the whole population, London: Springer-Verlag, pp 1–25. Davis, P., Marshall, R., Case, K., Porter, J.M., Summerskill, S.J., Gyi, D.E. and Sims, R.E., 2009. Reducing the stress of using public transport. In: Contemporary Ergonomics 2009, proceedings of the Annual Conference of the Ergonomics Society. London, UK, April 2009. DDA., 2005. Disability Discrimination Act 2005 [online]. Office of Public Sector Information. Available from: http://www.opsi.gov.uk/acts/acts2005/ukpga_ 20050013_en_1 [Accessed 01/09/2008]. Goodman, J., Dong, H., Langdon, P.M. and Clarkson, P.J., 2006. Industry’s response to inclusive design: a survey of current awareness and perceptions. In: P.D. Bust ed. Contemporary Ergonomics 2006, proceedings of the Annual Conference of the Ergonomics Society. Cambridge, UK, April 2006, pp. 368–372. Gyi, D.E., Porter, J.M. and Case, K., 2000. Design practice and ‘designing for all’, Proceedings of the IEA 2000/HFES Congress, Human Factors and Ergonomics Society, San Diego, California, USA, August 2000. Marshall, R., Case, K., Porter, J.M., Sims, R.E. and Gyi, D.E., 2004. Using HADRIAN for Eliciting Virtual User Feedback in ’Design for All’, Journal of Engineering Manufacture; Proceedings of the Institution of Mechanical Engineers, Part B, 218(9), 1st September 2004, 1203–1210. Martin, J., Meltzer, H. and Elliot, D., 1994. OPCS surveys of disability in Great Britain: The prevalence of disability among adults, London: HMSO. Oliver, R.E., Gyi, D.E., Porter, J.M., Marshall, R. and Case, K., 2001. A Survey of the Design Needs of Older and Disabled People, In: M.A. Hanson ed. Contemporary Ergonomics 2001 proceedings of the Annual Conference of the Ergonomics Society. Cirencester: Taylor & Francis, 2001, pp 365–370. Porter, J.M., Case, K., Marshall, R., Gyi, D.E. and Sims, R.E., 2006. Developing the HADRIAN inclusive design tool to provide a personal journey planner,
60
R. Marshall et al.
In: P.D. Bust ed. Contemporary Ergonomics 2006, proceedings of the Annual Conference of the Ergonomics Society. Cambridge, UK, April 2006, pp. pp 465–469. Porter, J.M., Marshall, R., Freer, M. and Case, K., 2004. SAMMIE: a computer aided ergonomics design tool. In: N.J. Delleman, C.M. Haslegrave, and D.B. Chaffin eds. Working Postures and Movements – tools for evaluation and engineering. Boca Raton: CRC Press LLC, 454–462. Summerskill, S.J., Marshall, R., Gyi, D.E., Porter, J.M., Case, K., Sims, R.E. and Davis, P., 2009. Validation of the HADRIAN System with a Train Station Design Case Study. In: Contemporary Ergonomics 2009, proceedings of the Annual Conference of the Ergonomics Society. London, UK, April 2009. WHO, 2008. WHO Ageing [online]. World Health Organisation. Available from: http://www.who.int/topics/ageing/en/ [Accessed 01/09/2008].
DEVELOPMENT OF TOOLS FOR REDUCING THE STRESS OF USING PUBLIC TRANSPORT P.M. Davis1 , R. Marshall1 , K. Case2 , J.M. Porter1 , D.E. Gyi3 , S.J. Summerskill1 & R.E. Sims1 1
Department of Design & Technology, Wolfson School of Mechanical & Manufacturing Engineering 3 Department of Human Sciences, Loughborough University, UK 2
Loughborough University’s initial role in the Accessibility and User Needs in Transport (AUNT-SUE) project led to the development of a database that describes 100 people. This drives a 3D human modelling tool called HADRIAN that allows designers and ergonomists to assess discrete interaction points in a journey. The need for both the public and practitioners to assess the accessibility of whole journeys has led to the development of two further tools. The Inclusive Journey Planner is based on the premise that reducing stress in anticipation of a journey will increase inclusion. It will show how existing internet journey planners could be improved to provide richer and more personalised information so that users are prepared for potential barriers and avoid needlessly stressful journeys. The Journey Stress Calculator provide a novel way for transport professionals to investigate transport accessibility improvements. It models the stress that the 100 HADRIAN participants would experience as a result of multivariate ‘stressors’ including having to climb steps or cross a road, navigating a crowded environment, stand on a bus, or purchase tickets.
Introduction AUNT-SUE A multidisciplinary team at Loughborough University has been part of the EPRSC funded Accessibility and User Needs in Transport (AUNT-SUE) project since its inception in 2004. AUNT-SUE was initiated to develop and test sustainable policies and practice that would deliver effective, socially inclusive design and operation in transport and the public realm. Now in its second phase; AUNT-SUE is producing tools that may be used by transport planners, designers of urban infrastructure and ergonomists to ensure that future developments improve accessibility. Tools being developed by AUNT-SUE include AMELIA, which provides guidance on improving practical accessibility based evidence collected in a geographical information system. It enables planners to compare possible solutions that increase access to 61
62
P.M. Davis et al.
important locations in a region. The Street Design Index also uses geographically collected data, but uses measures that are associated with perception of personal safety within a street environment to enable planners to pinpoint problem areas through which pedestrians will be unwilling to travel.
HADRIAN Loughborough University’s role in the first phase of AUNT-SUE led to the development of a database that describes 100 real people in terms of their size, shape, capabilities, behaviours, and experiences with transport and transport related tasks. As an alternative to using national population data; this database is intended to represent variation in humans and as such includes 59 people with many different forms of disability. The first implementation of this database is a design tool called HADRIAN. Based on the well established SAMMIE human modelling system, HADRIAN will allows designers and ergonomists to carry out virtual fitting trials using models of people in the database. In addition, a user-friendly version of the database including additional information, photographs and video clips will be available to help practitioners understand the behaviour of their subjects.
Whole journey assessment HADRIAN is suited to assessment of interaction points in a journey such as ticket barriers, lifts and vehicle ingress. However, there is also a need to assess the accessibility of the whole journey (Porter et al., 2006). From the perspective of individuals wishing to use public transport public; whole journey assessment is needed in order to find a journey that is best suited to their capabilities and prepare for anything that might make their journey difficult. From the perspective of practitioners; whole journey assessment is needed to compare journeys or problems within a journey in order to prioritise improvements. Two further tools are being developed to meet these needs. The Inclusive Journey Planner will build on the functionality of existing internet journey planners to assist individual users. The Journey Stress Calculator will provide practitioners with a new way to understand and assess whole journey accessibility by virtually sending each of the 100 HADRIAN Participants on any given journey. The challenge with both of these tools is taking into account not only features of the journey that physically prohibit travel, but also those that make participation difficult or unattractive. This challenge is being approached by considering the fundamental role that psychological stress plays in excluding individuals.
Stress and social exclusion The difficulty with assessing whole journey accessibility is that access is not simply achieved by removing ‘hard’ barriers that practically obstruct specific groups of disabled people. The current focus on social inclusion emphasises the need to ensure that people who are disabled, elderly, young, female, minority ethnic or low
Development of tools for reducing the stress of using public transport
63
socioeconomic status are not excluded. It is not always straight forward to identify the barriers that exclude these people. Many people cite unreliable service as a reason not to use public transport. This suggests that people can also be excluded by uncertainty due to a lack of knowledge or control over aspects of a journey. Within the AUNT-SUE project; the Street Design Index assesses ‘soft’ barriers such as lack of natural surveillance or vandalism to identify urban areas which people may be deterred from travelling through. While it is clear that these soft barriers are affect people psychologically; it is also possible to consider the psychological impact of hard barriers. Doing so means that all barriers can be considered in the same terms. Psychological stress theory provides the language for this.
Stress is a unifying concept The accepted model for psychological stress is the transaction between stressor and response (Cassidy, 1999). While major life events such as bereavement may be considered as stressors, daily hassles have been found to be better predictors of psychological symptoms and health. Such stressors may be described as acute, ambient or anticipatory. A hard barrier such as steps up into a vehicle will be an acute stressor for a person who has difficulty climbing steps. Soft barriers such noise or vandalism are ambient stressors because they can make a person feel at risk without presenting an immediate problem. Uncertainties such as not knowing what time a bus will arrive or which way to walk are anticipatory stressors because the person predicts that they may become late or lost. Considering all barriers as stressors allows comparison between practical, perceived and predicted problems with the stress response being the unifying measure of severity.
Stress is unfair The widely advocated social model of disability defines disabled people as those who face unequal difficulties in order to accomplish the same objectives as able people. While there is no reason to suggest that disabled people have a lower capacity to cope with stress; by his definition, disabled people will be exposed to more stressors as they participate in society. Getting on bus is more stressful if it is physically difficult. Wayfinding is more stressful if you have difficulty reading signs. Variation in the way people experience stress is not confined to people with medical disabilities. People of lowest socioeconomic status show greater signs of distress (Mirowsky & Ross, 1989). This has been attributed to greater exposure to severe stressors and fewer resources to aid coping in everyday life. Response to stress also varies with age; young adults show greater anxiety attributed to the stress of taking on new responsibilities, whilst older adults show greater depression attributed stress associated with deteriorating health. Other social patterns of distress relate to having children and the nature of employment. Such variation in people’s exposure to stressors and capacity to cope with stress explains why certain groups of people are more likely to find using public transport stressful and therefore be socially excluded.
64
P.M. Davis et al.
Stress excludes people Although a hard barrier, such as inability to walk up stairs, would stop some people travelling, there are many people who choose not to make public transport journeys for less clear-cut reasons. Such non-participation can be explained as a stress transaction as follows: A person anticipates that they could fail to complete a journey. This is anticipatory stressor and causes emotional distress. The person chooses to respond by flight from the activity. The person behaves in a way that enables them to avoid the journey. As such, anticipatory stress, based on predicted problems, has excluded the person. This means that it is not only the most severe stressors that need to be assessed. All stressors encountered in a journey, even if they are minor or unlikely, can contribute to the anticipatory stress that excludes people from travel. Therefore any stressor that can be eliminated from the public transport system is an opportunity to increase inclusion.
Stressor elimination is universally beneficial Reducing stress is not only important in terms of reducing exclusion. The elimination of stressors would benefit the wellbeing of all those who travel. When a person encounters a stressor that they cannot take direct action against (by fight or flight) then they will remain stressed. In activities such as travel where a person can encounter multitude of stressors that are beyond their control this stress will accumulate. Repeated or prolonged stress leads to deterioration of a person’s physical and psychological health. This is recognised in transport with Costa et al. (1988, quoted in Cassidy 1999, p. 50) finding increased psychological illness in commuters. As an approach to improving public transport accessibility; stressor elimination is attractive because it takes into account the full range of problems that can contribute to exclusion, acknowledges variation in people and has the potential to make using public transport a healthy and uplifting experience for all.
The Inclusive Journey Planner The Inclusive Journey Planner is intended to play a significant role in reducing the anticipatory stress that causes people to avoid travelling. It will do this by enabling users to find a route that will cause them least stress and give them the information they need so that they can assess whether the whole journey is accessible and reduce uncertainties as they go on the journey. While the Inclusive Journey Planner must be suitable for all, it should particularly benefit people with any of the difficulties shown in Table 1. The Inclusive Journey Planner will build on the functionality of existing internet journey planners such as Transport Direct, Traveline and the Transport for London
Development of tools for reducing the stress of using public transport
Table 1.
65
Inclusion of users with all kinds of disadvantage.
Circumstantial
Physical
Cognitive and Sensory
Inexperienced With Young Children Awkward Luggage Limited Money Limited Time No Car etc.
Limited Movement Wheelchair User Low Dexterity Low Strength Low Stamina Slow Walking etc.
Language Problems Learning Difficulties Hearing Impaired Vision Impaired Dyslexic etc.
(TfL) Journey Planner. Preliminary research was carried out to investigate how current UK journey planners provide travel information and how they might be improved upon. Significant variation in interface quality and level of information provided was found with the TfL Journey Planner outstanding in terms of the provision of accessibility options and information. User trials and structured interviews were then carried out with 14 people, including five with various disabilities. The trials picked up on usability issues and causes of dissatisfaction with the TfL journey planner. The interviews then explored opportunities to improve functionality and established some additional options and information that people would find beneficial. This process resulted in more than 60 recommendations that shall be implemented in the design of the Inclusive Journey Planner. The majority of these are encompassed by four main concepts: Personal Profiles, Genuine Journey Choice, Rich Journey Plans, and Better Walking Routes.
Personal Profiles Many existing journey planners have ‘Advanced Options’ which, for example, enable the user to specify preferred modes of transport and set a maximum walking time. However, most participants in the trials, including those who would benefit most from using these options, ignored them with the result that three disabled people found journeys that they were physically incapable of doing. Many of the options that are provided were found to be unnecessary, whilst new options such as change time allowance and choosing to avoid crowded or dark locations were popular. Personal Profiles should include only a limited range of useful settings so that users can choose preferences quickly before they proceed to entering their journey search. These would then be saved to ensure that people are given journeys that are suit to their needs every time without needing to re-enter information.
Genuine Journey Choice Current journey planners present a limited range of journey options to the user. These may be repetitions of the same route or a small number of similar routes at different times. Given that the user has specified a time of travel this is not the best way to provide choice.
66
P.M. Davis et al.
Figure 1.
Figure 2.
Personal Profile concept design.
Genuine Journey Choice concept design.
A Genuine Journey Choice system would group repetitions of the same journey as into a single journey option. The system would then present alternative routes that suit the user’s profile and enable the user to sort and filter these according to various criteria, including cost and walking time, so that they can always find their best possible way to make the journey.
Rich Journey Plans The TfL Journey Planner provides additional details beyond travel times, locations and vehicles. For many people this information, including service updates and the presence of steps, lifts and escalators, can be very useful and further information such as vehicle access, warnings about crowding and even weather predictions would be welcome. The challenge is to design such Rich Journey Plans without overloading the screen or the user so that people get the information they need to travel with confidence.
Better Walking Routes The final concept for the Inclusive Journey Planner goes beyond transport links to help people navigate through the urban environment. Whilst many existing journey
Development of tools for reducing the stress of using public transport
Figure 3.
67
Rich Journey Plan concept design.
planners provide route maps for walking between stops and stations, these are not tailored to the needs of the user. Providing Better Walking Routes means giving people accessible routes and information such as locations of pedestrian crossings, toilets, cash machines, steps and lifts. This may not only be done with conventional maps, but also text descriptions and local photographs.
Development, testing and outcomes The Inclusive Journey Planner will be developed as a web-based prototype with limited functionality to enable testing. Initial trials will be carried out to identify any usability problems before the final design is reached and tested against existing journey planners. The prototype will then be made available as an exemplar for future journey planners along with the results of the trials and guidance documentation. In addition to this openness, AUNT-SUE is actively seeking to assist transport organisations in the design of journey planners that reduce stress and promote inclusion.
The Journey Stress Calculator Given suitable options and the right information, individuals can make good judgements about whether they are able to go on a journey. For transport professionals however, it is very difficult to take into account the variation in the preferences and abilities of individuals when assessing the accessibility of different journeys. The Journey Stress Calculator is intended to solve this problem. It will make use of information about the 100 HADRIAN participants in order to model the level of stress that each individual is likely to experience as a result of encountering acute, ambient and anticipatory stressors in a journey.
68
P.M. Davis et al.
Figure 4.
Journey Stress Calculator showing variation in stress at different journey stages.
The stress calculator concept The stress calculator contains a library of stressors that may be encountered in journeys. For each of these stressors every participant has been assigned a stress response based on their ability to cope with that stressor on a scale where 0 = unaffected, 1 is mildly effected, 2 = moderately effected and 4 = significantly effected. The practitioner then determines the occurrence of each stressor at each stage in the process. For each individual, the total journey stress is simply the sum of all their stress responses. total_ journey_stress = (stressor_occurencen × stress_responsen ) The Journey Stress Calculator currently includes the responses of participants to 58 standard journey stressors. By specifying the combination of stressors that are present in the walking, interchange and vehicle stages, any given public transport journey can be uniquely modelled.
Tool development Development of the Journey Stress Calculator will follow three main strands. The first is the simplification of the journey inputting process. Ideally it would be able to take output of a journey planner directly as an input; however, practitioners will need to use their knowledge of the journey to determine the likelihood of some of the stressors. Inputting this information correctly requires a good user interface. The second strand is to develop different outputs. The Journey Stress Calculator can be used to judge whole journey accessibility and compare alternative journeys, but it can also provide in-depth information about who is excluded, why people
Development of tools for reducing the stress of using public transport
69
are excluded and identify particular stages in the journey that would benefit from investment. This means that results need to be presented in ways that suit different objectives. Thirdly, testing is needed to ensure that the results provide a realistic approximation of the actual stress that people experience in a journey. The way that stress is calculated may need to be modified to adjust the impact of different types of stressor. Once development is complete the Journey Stress Calculator should prove to be a practical tool and promote the understanding of exclusion in terms of the different levels of psychological stress that people experience.
References Cassidy, T. 1999, Stress, Cognition and Health, 1. (Routledge, London) Costa, G., Pickup, L., Di-Martino, V. 1988, A further stress factor for working people: evidence from the European Community, 1. A review. (International Archives of Occupational and Environmental Health), 60(5), 371–6. Cox, T. 1981, Stress, (MacMillan, London) Mirowsky, R. and Ross, C.E. 1989, Social causes of psychological distress, (Aldine de Gruyter, New York) Porter, J.M., Case, K., Marshall, R., Gyi, D.E. and Sims, R. 2006, Developing the HADRIAN inclusive design tool to provide a personal journey planner, Contemporary Ergonomics 2006 (Taylor & Francis, Cambridge), 470–4.
VALIDATION OF THE HADRIAN SYSTEM WITH A TRAIN STATION DESIGN CASE STUDY S.J. Summerskill1 , R. Marshall1 , D.E. Gyi3 , J.M. Porter1 , K. Case2 , R.E. Sims1 & P.M. Davis1 1
Department of Design and Technology, Department of Mechanical and Manufacturing Engineering, 3 Department of Human Sciences, Loughborough University, Loughborough, UK
2
The HADRIAN (Human Anthropometric Data Requirements Investigation & Analysis) human modelling system is under development at Loughborough University as part of the EPSRC funded AUNTSUE (Accessibility and User Needs in Transport for Sustainable Urban Environments) project. The HADRIAN system aims to foster a ‘design for all’ ethos by allowing ergonomists and designers to see the effects of different kinds of disability on the physical capabilities of elderly and disabled people. This system is based upon the long established SAMMIE system (System for Aiding Man Machine Interaction Evaluation), and uses data collected from 102 elderly and disabled individuals (Joint range of motion and anthropometry, ability to use steps and stairs, lifts escalators etc.). The HADRIAN system allows three dimensional CAD data of new products to be imported, with subsequent analysis using all of 102 sample members. The 102 sample includes a stature range of 1st% UK female to 99th%ile UK male, and also includes a range of disabilities that have been assessed using scales from Martin et al. (1994). In this way the needs of people with specific conditions, such as arthritis, can be demonstrated and where possible, design accommodation can be improved. This paper describes the validation activity that is underway with the HADRIAN system. The validation reflects the transport focus of the AUNT SUE project by using HADRIAN to analyse the user interaction points that people encounter when using the Docklands Light Railway in London. This includes the use of ticket machines, the use of the train station infrastructure such as lifts and steps and stairs, and the use of ATMs to obtain cash. The validation is being performed by comparing the predicted results from HADRIAN and the abilities of users when performing real life tasks such as retrieving a ticket from a machine, or pressing a floor button when in a lift.
Introduction Human modelling systems (HMS) such as JACK, RAMSIS, SAMMIE (Porter et al. 2004), and SAFEWORK are used in the design of vehicles, manufacturing environments and workstations. These systems use Computer Aided Design 70
Validation of the HADRIAN system with a train station design case study
71
Figure 1. The use of SAMMIE in the design of an automobile interior (2007). software to represent the size and shape variability of humans in simulations of environments such as car interiors (see Figure 1). The ability to simulate how people of different sizes and nationalities are accommodated by a product removes the need for costly early physical prototypes. If used correctly within a design process that includes later prototypes that verify the HMS analysis results, HMS can be highly cost effective. Currently HMS systems support design activity with a focus on able bodied people. The aging population in the UK (WHO 2008) and a greater awareness of the needs of disabled people (Disability Discrimination Act (DDA 2005)), have raised the prospect of using HMS to simulate the effects of disability, supporting the design of more inclusive products. Using human modelling systems to represent the effects of disability does raise issues in terms of the expertise of the end user. The designers and engineers that use HMS in the product design process generally have little experience of the effects of disability or the coping strategies used by disabled people. Current HMS systems have no methods of representing the coping strategies used by disabled people, such as sliding a kettle along a work surface instead of carrying it. This paper describes the validation of a new HMS system that has been designed to combine anthropometric, joint range of motion and behavioural data for a sample of disabled people. The aim of the new system is to foster a greater awareness of the effects of disability amongst designers and engineers, whilst providing a tool that supports a design process resulting in greater accommodation of the needs of elderly and disabled people. The test bed for the new system (HADRIAN) is the long established SAMMIE system (Marshall et al. 2004).
The HADRIAN system A key feature of the HADRIAN system is that the process of evaluating a product is automated, removing the product designer or engineer from key stages of the HMS
72
S.J. Summerskill et al.
process that require knowledge of the behaviour of disabled people. The HADRIAN system allows a product analysis to be performed on the basis of a task description provided by the software user. For example, if a ticket machine is to be evaluated, the user would import a CAD model of the ticket machine into the HADRIAN system and build a list of tasks to be performed. Example tasks for a ticket machine include the use of a control to select on screen options, and depositing money into the coin slot. The HADRIAN system can perform these interactions for all of the sample members built into it, using data on the coping strategies of each individual to automatically perform tasks such as the positioning of the virtual user, to allow the best reach to the various controls built into the product. The system can then identify which users were unable to complete certain task stages based upon their ability to reach and view the interaction points of a product. This allows design changes to be identified that can increase the accommodation of the product, such as lowering a control or changing a screen angle. The behavioural data that is used in HADRIAN was captured from the video recording of kitchen based activities of daily living performed by each participant. The system also contains demographic information such as the type of disability, age, sex and occupation.
The HADRIAN sample of users HADRIAN is based upon data collected from a sample of 102 people, the majority of whom were registered as disabled, or had age related impaired mobility. The sample of elderly and disabled users participated in the following data collection activities; anthropometric data, joint range of motion data, reach range data, completion of a questionnaire detailing the use of different modes of public transport, and the collection of baseline data on the ability of the participants to perform kitchen based activities of daily living. Within the sample of 102 people 59 people have some form of impairment including: limb loss, asthma, blood conditions, cerebral palsy, epilepsy, head injuries, multiple sclerosis, arthritis, vision and hearing impairments, heart problems, paraplegia, Parkinson’s disease, stroke, and dyslexia, amongst others. Of the 43 able bodied people 20 were aged over 60 and had undiagnosed or minor impairments associated with being older. The remaining participants provide baseline information on the capabilities of non disabled people. All of the sample members included in HADRIAN were capable of living independently. Each subject was assessed using a modified version of the OPCS sample frame (Martin et al., 1994) to allow a comparison of the severity of disability exhibited by the HADRIAN sample to prevalence and severity of disability in the UK.
Anthropometric data used in the HADRIAN subject simulations The following anthropometric measures were collected from each participant. Stature, Arm length, Upper arm length, Elbow-shoulder, Abdominal depth, Thigh depth, Knee-hip length, Ankle-knee length, Ankle height, Foot length, Sitting
Validation of the HADRIAN system with a train station design case study
73
Figure 2. The postures adopted by participants when performing kitchen based tasks. height, Sitting shoulder height, hip-shoulder length, Chest height, Chest depth, Head height, Eye-top of head, Buttock-knee length, knee height, Shoulder breadth, Hip breadth, Hand length and Grip length.
Joint constraint data used in the HADRIAN subject simulations The following joint constraint data measures were collected from each participant; shoulder extension, shoulder flexion, shoulder abduction, shoulder adduction, arm extension, arm flexion, arm abduction, arm adduction, arm medial rotation, arm lateral rotation, elbow extension, elbow flexion, elbow pronation, elbow supanation, wrist extension, wrist flexion, wrist abduction, and wrist adduction.
Data collected on positioning and posture The prototype version of HADRIAN containes automation data based upon the kitchen tasks that were performed in the user trials. The participants were asked to move a variety of objects onto a high shelf, a work surface, and into cupboards and shelves of standard kitchen units. This process was video recorded to allow the postures that were adopted to be coded (see Figure 2). Table 1 shows the positioning and postural data that were captured for both ambulant and wheelchair using participants. These coded data were used to inform the behavioural aspects of the HADRIAN task automation. A more detailed description of the HADRIAN system can be found in Marshall et al. (2009).
The validation of the HADRIAN system The HADRIAN validation process aims to verify and potentially improve the data that drives the automation of the product assessment process. This is done by comparing the results of the analysis of the accessibility of three travel interaction
74
S.J. Summerskill et al.
Table 1. The coding information used to classify the postures exhibited by the HADRIAN sample members during the kitchen tasks. Postures and orientations to be coded Orientation of the user to the kitchen cupboards Arm used for the tasks The posture of the legs during the kitchen tasks (ambulant participants only) Back twist Back bend Shoulder Head orientation
Coding criteria for use in HADRIAN Face on, side on, angled approach Left or right Straight, bent 1 (knee angle 170–120◦ ), bent 2 (knee angle 119–40◦ ), crouch (knee angle 39–0◦ ), left kneel, right kneel, full kneel, sitting Left or right >10◦ Upright (0–10◦ ), lean (11–45◦ ), bend (46◦ +) Relaxed, extended Yaw (neutral, left, right +/− >10◦ ) Pitch (neutral, forward back +/− >10◦ ) Tilt (Neutral, left, right, +/− >10◦ )
points, namely, a ticket machine, a lift at a railway station and an ATM machine. The selection of travel interaction points reflects the aim of the AUNT SUE project, which is to reduce social exclusion by supporting increased accessibility to the travel infrastructure by elderly and disabled people. Three different analysis techniques are used to assess the interaction points. Firstly an expert user of HMS, with extensive experience of the coping strategies used by disabled people, performed an analysis using HADRIAN data in SAMMIE. The SAMMIE system allowed the expert HMS user to simulate the anthropometry and joint range of motion only, with the experience of the expert user driving the behavioural aspects of the product analysis. The second analysis technique involves the use of the HADRIAN system to perform an automated analysis. Finally, a selection of the HADRIAN sample are asked to perform the same product interactions that were performed by their virtual counterparts with the real products. This has the potential to illustrate unexpected coping strategies or task failures that were not anticipated in the virtual user trials performed in SAMMIE and HADRIAN. Any new information gained regarding user behaviour can then be applied to refine the automation of HADRIAN for specific tasks.
Validation sample members Ten HADRIAN subjects will participate in the validation process. The subjects selected were two people over 65 with arthritis (male and female), an ambulant disabled female with cerebral palsy who uses a wheeled walking frame, a male mobility scooter user with mobility problems caused by a brain tumour, a powered wheelchair user with limited strength in the right arm due to a stroke, and five manual wheelchair users with a range of upper body mobility. These participants exhibit physical limitations in joint range of motion and reaching ability that affect their ability to interact with products, which can be compared between the three analysis techniques.
Validation of the HADRIAN system with a train station design case study
75
Figure 3. The analysis of an NCR ATM design with a paraplegic participant with good upper body mobility.
Validation progress The analysis of the lift, ticket machine and ATM has been performed by the expert user with the sample of ten HADRIAN participants. Figures 3, 4 and 5 show the analysis in progress for the ATM, ticket machine and lift respectively.
The analysis of two NCR ATM designs The cash machine manufacturer and project collaborator NCR has provided two cash machine designs for the validation process. This includes real ATM fascias for an analysis using the HADRIAN sample members, and CAD data to allow a virtual analysis using HADRIAN and SAMMIE as shown in Figure 3. The ATM illustrated in Figure 3 is a new design that is currently being installed world wide. An earlier model has also been provided by NCR. The analysis of the ATM involves different mounting heights for the ATM, the range of which is found internationally. Each HADRIAN participant will be asked to perform typical product interaction stages (insert a credit card, select a cash withdrawal by using a screen button, collect the cash, deposit a cheque in an envelope) which have been reproduced using the virtual analysis techniques.
The analysis of the ticket machine used by the Docklands Light Railway (DLR) in London The analysis of the ticket machines used by the DLR will be performed using a sample of participants based in the London area. These participants have been
76
S.J. Summerskill et al.
Figure 4. The analysis of the ticket machine used in DLR stations in London with a participant that has limited mobility due to a stroke.
matched to members of the original sample in terms of their type and severity of disability. This allows an analysis of the general applicability of the design recommendations made using the original HADRIAN sample members, when compared to other people with similar conditions. For example, the participant shown in Figure 4 exhibits limited shoulder rotation and arm strength due to a combination of a broken back suffered in a car accident and a subsequent stroke. This meant that the joint constraints applied to the human model in SAMMIE and HADRIAN indicated difficulty in reaching to the controls and coin slot incorporated in the design. This necessitated a near sideways approach by this wheelchair user which allowed the virtual participant to reach all of the interaction points, but also caused a poor viewing angle to the screen. The completed virtual analysis determined that the user would find it difficult to read the on screen information. The comparison between the virtual results and those of the real participants will be the basis for the validation HADRIAN data for this product. The virtual analysis has been completed in both the HADRIAN and SAMMIE systems, providing a range of design recommendations that would improve the accommodation of the device for use by elderly and disabled people, whilst not disadvantaging non disabled users. The next stage of the validation process will be the matched sample user trials in London, which will follow a scenario that includes the use of the ticket machine, accessing the platform level using the lift or a ramp that is over 100m long, boarding a train and then returning to the original station. The DLR station is due for refurbishment. The DLR management have supported the validation process by providing CAD data for ticket machines and access to the Greenwich station for measurement and assessment activities. The design recommendations produced by the validation process will be considered by the DLR management for inclusion in the refurbishment of the Greenwich DLR station.
Validation of the HADRIAN system with a train station design case study
77
Figure 5. The analysis of a lift found at the Greenwich Docklands Light Railway (DLR) station in London with a participant with mobility problems caused by a brain tumour.
The analysis of a lift used in the DLR train station in Greenwich, London The analysis of the lift used in the DLR train station will follow the matched sample approach used in the analysis of the DLR ticket machine. The virtual analysis of the lift has indicated a number of design recommendations. The virtual evaluation shown in Figure 5 indicated that a mobility scooter user would be unable to share the lift with other users due to space constraints. Also, the limited mobility of the user indicated difficulty in looking backwards when reversing the mobility scooter out of the lift (the user had removed the mirrors usually attached to the mobility scooter). Other HADRIAN sample members indicated that the limited width of the lift would make it difficult for people with limited upper body mobility to access the buttons for the alarm and holding the doors open, as these were located in the highest location on the control panel. The matched sample that will be asked to use this lift at the Greenwich DLR station will be observed to determine if these issues arise.
Conclusions The HADRIAN validation process has been designed to verify and improve the automation of the HMS analysis of products that are associated with the travel
78
S.J. Summerskill et al.
infrastructure. The virtual analysis that has been completed has highlighted access issues that could hinder the use of public transport systems by people with disabilities and older people. The verification of these results with real users will demonstrate the effectiveness of the HADRIAN system for the analysis of tasks which are different to those upon which the automation is based, i.e. kitchen based tasks. Any differences in the assumed behaviour of the virtual HADRIAN sample members that are found in reality, will serve to fine tune the automation of the HADRIAN system. The dissemination of the HADRIAN dataset will take three forms. The first form will be a database that presents the information gathered to researchers and designers. The second form will be data which can be applied to all current HMS systems in terms of anthropometry and joint range of motion, and the third form will be HADRIAN itself, which will combine all of the data available with automation of the product analysis. The use of HADRIAN has the potential to support designers and ergonomists in their efforts to improve access for disabled people who have the capability to live independently, but who are excluded from public transport use by an infrastructure design process that does not currently take account of their needs.
References Disability Discrimination Act 2005 [online]. Office of Public Sector Information. Available from: http://www.opsi.gov.uk/acts/acts2005/ukpga_20050013_en_1 Marshall, R., Case, K., Porter, J.M., Sims, R.E. and Gyi, D.E., 2004. Using HADRIAN for Eliciting Virtual User Feedback in ‘Design for All’, Journal of Engineering Manufacture; Proceedings of the Institution of Mechanical Engineers, Part B, 218(9), 1st September 2004, 1203–1210. Marshall, R, Summerskill, S.J., Gyi, D.E., Porter, J.M., Case, K., Sims, R.E. and Davis, P., 2009. A design ergonomics approach to accessibility and user needs in transport. In: Contemporary Ergonomics 2009, proceedings of the Annual Conference of the Ergonomics Society. London, UK, April 2009. Martin, J., Meltzer, H. and Elliot, D., 1994. OPCS surveys of disability in Great Britain: The prevalence of disability among adults, London: HMSO. Porter, J.M., Marshall, R., Freer, M. and Case, K., 2004. SAMMIE: a computer aided ergonomics design tool. In: N.J. Delleman, C.M. Haslegrave, and D.B. Chaffin eds. Working Postures and Movements – tools for evaluation and engineering. Boca Raton: CRC Press LLC, 454–462. WHO, 2008. WHO Ageing [online]. World Health Organisation. Available from: http://www.who.int/topics/ageing/en/ [Accessed 01/09/2008].
COMPLEX SYSTEMS
A SOFTWARE TOOL FOR EVALUATING THE EFFECTS OF CULTURE ON MILITARY OPERATIONS A. Hodgson & C.E. Siemieniuch Loughborough University This paper presents the military rationale for developing an improved understanding of the effects of culture on the performance of military organisations and their partners. In particular the change from conventional high intensity warfare to asymmetric warfare increases the interactions between different cultures within multi-national military forces and in the local populations. A brief description of culture and cultural attributes is presented. A culture-based evaluation tool (the ‘Soft Factors Modelling Tool’), which has been developed by the authors, is described.
Introduction The paper reports on work funded by the UK Defence Technology Centre Systems Integration and Integrated Systems for Defence: Autonomous and SemiAutonomous Systems. The focus of the project is on the configuration of complex military systems (which will comprise various combinations and configurations of technical and non-technical [ie human] components). There is an increasing recognition of the potentially deleterious effects of incompatible individual and organisational cultures on complex systems and organizations. This applies in both military and civilian environments. In the case of the military, these effects are exacerbated by the increasing involvement of multinational forces and nongovernmental organizations as the world moves into an era of ‘asymmetric warfare’, explained below. A basic premise of the project is that these systems need to exhibit a range of desired behaviours commensurate with the environment within which they are operating, the tasks they have been set and the degree of autonomy desired. It is anticipated that the research will enable a greater understanding of the impact that different configurations of cultural attributes can have in facilitating or impeding systems in making, communicating and implementing decisions, including the requirements for organisational change.
Military rationale for research into a cultural assessment tool Major changes are occurring to the post Cold War environments in which Western armed forces are operating. These changes are described briefly in this section. 81
82
A. Hodgson & C.E. Siemieniuch
The increasing frequency of asymmetric warfare In military terms, the USA is now virtually unchallengeable in conventional high intensity warfare capability. However, this hegemony, by limiting the military options of would-be foes, has forced them to select alternative, ‘asymmetric’ or ‘complex irregular warfare’ approaches (International Institute for Strategic Studies, 2005). Western strategists have realized that the West will not only face asymmetric warfare threats from terrorists based in ‘failed states’, but also from developing countries which have already tested and evaluated asymmetric warfare capabilities (Schneider 2007). It is generally accepted that it will be necessary to project force wherever the perceived enemy is detected.
A need for changes in Western military forces The Western soldier will in future operate increasingly in vulnerable situations, where he or she is always effectively on the front line, facing surprise attack, hostage-taking and arbitrary changes in local alliances amongst armed groups who use the civilian population to shield themselves. It is important to reduce the risks that this soldier faces, wherever possible; this risk reduction will come about via more appropriate training, cultural awareness, improved weaponry and physical protection and improved information. Based on more than fifty years experience of anti-terrorist activities at home and abroad, the UK Ministry of Defence is well aware of many of the issues associated with low-level asymmetric warfare, and while much greater reliance must, in the future, be placed on unmanned systems for surveillance, communications, targeting and as weapon platforms, there will always be the need for ‘feet on the ground’ in future conflicts.
The project It is recognized within the SEAS DTC that personal culture and organizational culture have a significant effect on the effectiveness of military units. Therefore, the DTC is funding research projects covering cultural issues – the work of one such project, Impact of Different Cultural Attribute Sets on Semi/Autonomous System Decision Structures and Interfaces, is the main focus of this paper. One of the first tasks of the above research project was to carry out a major review of the literature related to culture and its effects, in particular with regard to the military (Siemieniuch & Meese 2006). The outputs of this survey covered twenty-four topics, including: • • • •
What is culture? National culture Professional culture Organizational culture
A software tool for evaluating the effects of culture on military operations
• • • •
83
Military culture Cross-cultural operations Transforming culture Cultural dimensions
These outputs provided an essential guide to the subsequent activities of the research project, in developing a cultural evaluation tool.
Military scenarios of interest There are five key areas where culture has a particular role to play; these are described below: 1. Increasingly powerful soldiers: Advanced autonomous and semi-autonomous systems place greater power than ever before in the hands of individual soldiers, and it is important that they make appropriate decisions in using this power. Individual, group and organisational culture play a significant part in such soldiers’ effectiveness. 2. Effects-based operations and force packaging: UK and US forces are moving towards ‘effects-based operations’, where performance measures are based on the achievement of objectives, not the provision of fire power. ‘Force packaging’ (modular, self-contained sets of units borrowed from the various commands for a given mission) is an important contributor to this approach. Force packages will be assembled at short notice from different service cultures to form a force appropriate for an unforeseen crisis. 3. Multinational and non-governmental organisations: Multinational forces are increasingly being deployed, typically working alongside non-governmental organisations (NGOs) and operating in conflict, peace-making and peacekeeping roles. However, the organisations operate within different cultural contexts and exhibit a range of cultural attributes which affect cohesion and increase the likelihood of disagreements and misunderstandings. 4. Local populations: Military forces are spending ever longer periods in foreign countries. To achieve ‘success’, they must interact in culturally sensitive ways with the local populations of the host countries in order to go from crisis to communal action. 5. The development of semi/autonomous systems: Designers make their own culturally-biased assumptions about user requirements and expectations. Organisational and military research shows that technology tends to be adopted by users for their own purposes, and this is heavily influenced by the extant organisational culture.
Cultural perspectives Culture is a major and underestimated factor in the performance of any complex system. Cultural effects can be examined at the level of the individual, the team,
84
A. Hodgson & C.E. Siemieniuch • Unique • Part genetic • Part learned, e.g. individual behaviour, values
INDIVIDUAL
COLLECTIVE
UNIVERSAL
Figure 1.
• Shared • Learned, e.g. national, professional & organizational culture • Common • Genetic • Pre-programmed, e.g. laughing, weeping, aggression
Hofstede’s perspectives on culture.
and organisation; however, culture also affects the design of systems (whether or not they contain humans). Hofstede’s (1984) definition of culture is regarded by the research team as a useful ‘working basis’: Culture is “the collective programming of the mind which distinguishes the members of one human group from another . . . includes systems of values; and values are among the building blocks of culture.” Hofstede (1991) provides three broad perspectives on culture, as depicted in Figure 1. The perspectives that are amenable to change are at the individual and the collective culture levels. Hofstede uses the term ‘cultural values’ where value is seen as ‘a broad tendency to prefer certain states of affairs over others’. In this project the term ‘cultural attributes’ has been used to describe a set of high level culturerelated features, the values of which can be used to describe individuals and systems (including technical systems).
National culture National culture is usually a product of heritage (religion, history, language, climate, population density, availability of resources, politics, etc.). The following can vary according to national culture and can therefore shape expectations and performance: • • • • •
Leadership styles (hierarchical vs. consultative) Superior – inferior relationships (accept vs. question decisions) Communication styles (direct and specific vs. indirect and non-specific) Emotional reaction (showing reaction, emotion or aggression vs. hiding reaction) Following vs. breaking rules
A software tool for evaluating the effects of culture on military operations
85
Professional culture Professional culture is usually manifested in its members by a sense of community and by the bonds of a common identity (Helmreich & Merritt, 1998). Attributes of professional culture can include: • • • • • • • • • • •
Members have specific expertise and a shared professional jargon Norms for behavior and common, binding ethical values Selectivity and competition for entry Prestige and status with badges or defining uniform Extensive training requirements Professional and gender stereotyping Status differentials Self-regulation Institutional and individual resistance to imposed change Reluctance to admit error and denial of ‘vulnerability’ Reduced awareness of personal limitations
Organizational culture There are two layers of organizational culture: • Formal, surface, visible structures e.g. uniforms, routines, documents, etc. • Informal, inner, invisible layers, e.g. values, beliefs, subconscious assumptions. • Organisational culture is more amenable to influence than professional or national culture and it is organisational culture that effectively channels the effects of the other two cultures into standard working practices. Organisational culture is specific; what works in one organization may not work in another, and may not work at a different branch of the same organization. • Factors thought to influence or engender organizational culture include: – Reduced awareness of personal limitations – Strong corporate identity (e.g. nature of the product and market branding) – Effective leadership – belief in the organisation’s mission and products – High morale and trust – confidence the organisation’s practices, management, communication and feedback – Cohesive team working and cooperation – Job security – Development & training – Degree of empowerment
Cultural attributes In contrast to much work on culture, some cultural attributes described in this section can be exhibited by technical elements such as intelligent software agents. Since systems comprise both technical and non-technical elements, the technical components of semi/autonomous systems will also need to demonstrate appropriate decision-making behaviours and an ability to perform in particular environments.
86
A. Hodgson & C.E. Siemieniuch
Attributable to human agents Individualism
Collectivism
Universalism
Particularism
Hunter-gatherer Power by achievement Mastery
Nurturer Power by status Fatalism
Attributable to human & technical agents Proactive Information integration Time synchronization High power distance High risk-taking
Orthododox Information analsis Time sequencing Low power distance Low risk-taking
Figure 2. Cultural attribute pairings. For brevity, only one of these pairs is described. Further details are provided in Johnson et al,. (2007).
Technical sub-systems or agents are designed by humans, and it is inevitable that positions on the cultural attribute pairings will, in many cases be inherent in the design of these technical subsystems, in particular where there is intelligence in the sub-system.
Overview of the chosen cultural attributes In this study Culture is currently assessed based on ten different pairings of cultural attributes. of which five can only be exhibited by the human agents in a system; the other five attributes can be exhibited by both human (i.e. non-technical) and technical (i.e. software-based) agents within a system, as indicated in Figure 2. The project researchers believe that there is an identifiable relationship between the configuration of cultural attributes exhibited by a system (comprising both human and technical agents and components) and the performance of that system in a particular environment. Cultural attributes can relate to the perceptions of a single agent (e.g. an individual or missile launcher), a group of sub-systems (a troop or a communications infrastructure) or an overall system (the army or the set of assets carrying out a mission). Each attribute pairing defines a range, anchored by a description of the likely beliefs, perceptions, etc., at each end. Individuals, groups or systems will select a position towards one end or the other. It should be remembered that the attribute ranges, of themselves, do not imply right or wrong, merely that the attribute value is more or less appropriate given a required level of performance in a particular environment.
A software tool for evaluating the effects of culture on military operations
87
Individualism vs. collectivism: This refers to the balance struck between individuals and groups. In individualistic societies, ties between individuals (other than immediate family members) are loose; each person takes responsibility for his or her actions; individualists tend to speak directly and factually, and are willing to argue and to question others’ views. Generally, individualists tend to exhibit a higher level of trust than collectivists. In collectivistic societies, individuals are integrated into closely knit groups, often in the form of extended families; in return for unquestioning loyalty, they gain the protection of their group. People from outside the group tend to be treated with suspicion. Collectivists try to avoid direct, confrontational approaches and find criticizing others difficult. Hierarchies tend to be rigid, and losing face is to be avoided at all costs.
The Soft Factors Modelling Tool The latest version of the Soft Factors Modelling Tool (SFMT) enables individuals and representatives of subsystems (teams, squads, platoons, etc.)to input estimates of their positions with regard to each of the cultural attribute pairings. These positions are converted into numeric values and feedback is presented to the user(s) graphically to show conflicts between cultural attributes at the various system levels. The tool will highlight: • Any similarities or conflicts in cultural attribute values that exist between the various assets • Areas where particular assets may show a cultural attribute value that will facilitate or inhibit a desired behavior or ability to operate in a selected environment The target end-users of the final version of the tool are mission planners who are required to put together a package (comprising human and technical components) to carry out an operational requirement in a particular environment. For example the tool can help answer the question ‘Is this particular configuration of military assets with this mission capable of demonstrating appropriate decision-making, information processing, communication, adaptive skills and behaviour in an environment where the command style is control free, authority is delegated, operational tempo is unpredictable and the battle space is ill-defined?’
Impact of cultural attributes on behaviour As indicated earlier, it is believed that exhibiting certain combinations of attribute values will facilitate or inhibit certain types of decision-making behaviour. For example if there is a required behaviour for processing information which included the ability to deal with ambiguity and contradictions in the information or to deal with incomplete information, it is evident that an agent or system demonstrating a position towards the analysis end of the analysis v synthesis spectrum would find this difficult. Within the SFMT the cultural attributes have been evaluated
88
A. Hodgson & C.E. Siemieniuch
Table 1.
Part of behaviours checklist.
Skill class
Desired behavior
2nd Lt. Smith Troop
Regiment
Communication & interaction skills
Say what you mean/mean what you say Convey meaning indirectly diplomatically Dispel conflict Ability to trust and be trusted Willingness to collaborate/ co-operate Transparency/openness
POS
POS
UNCLR
NEG
UNCLR
POS
POS POS POS
POS POS UNCLR
POS UNCLR POS
POS
POS
UNCLR
Process information rapidly
POS
POS
UNCLR
Deal with ambiguity Deal with complexity Deal with contradictions Deal with uncertainty Deal with incomplete information Objective analysis of technical data Prioritize information Sharing information
NEG UNCLR UNCLR UNCLR UNCLR
UNCLR POS UNCLR POS UNCLR
POS UNCLR POS UNCLR POS
UNCLR
POS
POS
UNCLR POS
POS POS
POS NEG
Information processing
against a set of desired behaviours and a simple scoring system developed Table 1 illustrates a section of a spreadsheet that lists on the left a range of skill classes and, within these, behaviours; note that some of these are mutually exclusive, some are complementary. Cultural attribute pairs are listed across the spreadsheet and scored, although only one of these pairings, individualism vs. collectivism, is shown in Table 1 due to space limitations. The third and fifth columns of Table 1 indicate the capabilities of collectivism and individualism to achieve the various behaviours. Differing combinations of behaviours would be required for effective decision-making in various combat, peacemaking and peacekeeping situations. The analysis engine evaluates the values input for the cultural attribute pairs against each of the classes of behaviour of Table 1, using a simple traffic light display system. Note that, for this paper, the green, amber and red colours in Tables 1 have been replaced with the words POS (positive), UNCLR (unclear) and NEG (negative).
Impact of cultural attribute values on the ability to perform in particular environments Military environments vary considerably in terms of climate, terrain, command structures etc. The cultural context, within which military operations take place, also varies.
A software tool for evaluating the effects of culture on military operations
Table 2.
89
Extract from sample analysis results for behavior.
Environment class
Class characteristics
Command structure and style
Centralised Structure POS Decentralised Structure NEG Authoritative/interventionist Collaborative/control free POS Formal communication structure Informal communication structure Strong directive leadership Consensus based leadership POS
Function/authority/ Stove-piped function skills distribution distribution Dispersed function distribution Centralised authority over actions Delegated authority over actions Specialist skill set available Multi-skilling predominates
Collectivism
No obvious tendency
Individualism NEG POS
UNCLR
UNCLR
POS
UNCLR
POS
NEG
NEG
POS
POS
NEG
NEG
POS
As stated earlier, agents or systems exhibiting certain positions on the cultural attribute pairings will inhibit or facilitate agent or system performance in a given environmental context exhibiting certain characteristics. Another table, similar to Table 1 captures the environmental characteristics.
Presentation of SFMT results A final table provides the results of the analysis, after comparisons between the Tables above, providing an assessment of the goodness of fit between the proposed team and the environment – see Table 2 below. It has the same kind of structure, with a traffic light system to show distinct incompatibilities. The user may then decide whether there are too many reds and ambers that the plan must be reconsidered, or to go ahead anyway, or adopt some other alternative.
Further work Following completion of the current version of the SFMT, it will be evaluated by a wide range of military and other scenarios, prior to the development of a final version and recommendations for additional work. Many issues are still to be resolved, for example: • What are the optimum sets of cultural factors and behavioural/skill classes that achieve comprehensiveness without confusing overlaps? • Are the environmental factors too environment-specific to be transferable across environments?
90
A. Hodgson & C.E. Siemieniuch
• Who should input the soft factor values? • Could this tool be used to good effect in civilian scenarios?
References Helmreich, R.L. and Merritt, A.C. 1998. Culture at Work in Aviation and Medicine. Aldershot, UK, Ashgate. Hofstede, G. 1984. Culture’s Consequences: International Differences in WorkRelated Values. Beverley Hills, Sage. Hofstede, G. 1991. Cultures and organisations. London, McGraw-Hill. International Institute for Strategic Studies. 2005. Complex Irregular Warfare: The Face of Contemporary Conflict. In Military Balance, 105(1): 411–420. Johnson, P., Siemieniuch, C.E. and Woodhead, M.A. 2007. Description of 1st Prototype Version of Soft Factors Modelling Tool, Technical Report, SEICRP-0605, SEIC, Dept. of Electronic & Electrical Engineering, Loughborough University, UK. Schneider, W. 2007. Asymmetric Military Aspirations and Capabilities of the People’s Liberation Army of the People’s Republic of China. US-China Economic and Security Commission, Washington DC, March 2007. Can be downloaded from: http://www.uscc.gov/hearings/2007hearings/transcripts/mar_29_ 30/schneider.pdf Siemieniuch C.E. & Meese N.C. 2006. Technical Deliverable Report SEIC-RP520, SER006, SEIC, Dept. of Electronic & Electrical Engineering, Lougborough University, Leics, UK.
DECISION-MAKING AFTER THE PRODUCT-SERVICE SHIFT AND SOME IMPLICATIONS FOR ERGONOMICS E.M. Molloy, M.A. Sinclair & C.E. Siemieniuch Loughborough University, UK This paper introduces a Decision-Making System Framework developed in the UK EPSRC-funded Grand Challenge, entitled ‘Knowledge and Information Management – through life’ (KIM), for the evaluation of decision-making processes and systems. The development of the framework is outlined, along with a brief outline of the validation work. The use of the framework will be discussed, with reminders that the goal of this framework is not to provide the ‘right’ answer, which is improbable, if not impossible, to predict when dealing with such extended and complex systems.
Introduction A significant challenge, particularly for engineering organisations, is the ‘productservice shift’, whereby organisations move from delivering a product to the provision of through-life, sustained capability support (Oliva and Kallenberg 2003; MOD 2005; Anon 2008). This affects companies in their internal organisation and requires changes to the processes that they employ; particularly for knowledge and information management. Furthermore, the problems will get steadily worse as systems evolve from standalone status to components of systems-of-systems, with all the issues associated with this shift. While this happens, the making of complex decisions will remain a human and organisational issue. This is the context of this paper. The paper outlines the background of decision-making systems (DMS) and complexity before outlining the development and subsequent validation of a DMF (Decision Making system Framework) and supporting process. It builds on work initially presented at the ErgonomicsAnnual Conference 2008 (Molloy et al., 2008).
Organisational decision-making While there has been a considerable literature on Decision Support Systems, there appears to be a dearth on the characterisation and evaluation of DMSs that may use DSSs; Simon (1997) is one example. This is a pity; from an ergonomics perspective; 91
92
E.M. Molloy, M.A. Sinclair & C.E. Siemieniuch
as Rasmussen (1997; 2000) has said, “A closer look at the recent major accidents show that they are not caused by a stochastic coincidence of failures and errors, but by a systemic migration of organisational behaviour towards the boundaries of safe operation. Major accidents are the side-effect of decisions made by several decision-makers, in different organisations, and at different points in time, all doing their best to be effective locally.” One of the underlying causes of this is complexity: “A system exhibits complexity if it is composed of many integrated entities of heterogeneous parts, which act in a coordinated way and whose behaviour is typically nonlinear.” (Rycroft and Kash 1999). Some organisational characteristics which give rise to this are (Gregg 1996): • • • • • • •
many agents many kinds of agents operating in parallel with a degree of behavioural autonomy non-linear interactions multiple steady states for the system multiple steady states for human agents
As Gregg says, not many of these are needed for the system to exhibit emergent behaviour. A personalised view of this (Vaill 1998) is: “Furthermore, there was the problem of the intrusiveness of events: things did not occur one at a time; no competence could be practiced in pristine singularity. Instead, at any moment I was flowing with the multiple, disjointed time streams of the various projects in which I was involved.. . . Everything was interactive. . . . I simply had to learn to understand myself in a spatio-temporal field of relationships, flowing and shifting. It was a field of multiple players, each of whom had his or her own schedules, expectations of the rates at which things ought to proceed, and resistances to being sidetracked by other people’s temporal perceptions and priorities.” This is the context of DMS; seldom is it discussed in the literature, but it is the problem that we chose to research, because of its relevance to practitioners of systems ergonomics.
Decision-making systems and decision support systems It is well-known that decision-making is affected by external environmental and commercial pressures, and by internal effects and pressures. Many of these, such as established processes, structures, and incentivisation policies, are determined by the company’s overall strategy. It is important that strategy is developed and implemented in an arena of good governance, otherwise the endeavour to make good decisions will be vitiated and detrimental emergent behaviour will be a nearcertainty.
Decision-making after the product-service shift and some implications
93
Decision-Making Systems (DMS) Included within the term DMS as it is used within this paper are: • Agents – software or human based, who are involved with decisions • Activities – the decision-making activities which enable decisions to be made • Infrastructure and technology – to facilitate decision-making, including computer based support • Knowledge and Information. Its distribution across the organization (the ‘knowledge configuration’) determines the effectiveness of decisions made. DMS will of course be affected by time (for example, time available in which to make the decision, time by when the decision must be made, time when the effect of the decision is realized and expected duration of the decision) and the style of decision-making.
Decision Support Systems (DSS) In this research, DSS are considered to form part of the overall DMS. In much of the literature (Silver, 1991; Finlay, 1989) DSS are considered to include only computer based tools. However, this research widens that definition to include any form of support or guidance, which may be computer based or from a more human source, such as Communities of Practice (CoP) (Coakes and Clarke, 2006; Wenger, 2000).
Development of a decision-making systems framework The aim of the DMS framework and the associated process of use is to assist organisations in configuring their DMS to help them to cope with the risks and opportunities of long life, complex, engineered projects and systems. A top level view of the framework is shown in figure 1; lack of space precludes more detail. The framework has been developed from meta-analysis of accident and incident reports, pilot studies and industrial based case studies, all of which embrace complexity in some way. These are outlined below.
Literature studies The incidents meta-analysed were predominantly accident cases where a wealth of information and documentation is available (e.g. Enron, Piper Alpha). These studies investigated the decision processes and issues leading up to the various incidents, not the causes nor the blame. The analyses identified commonalities between the incidents; for example, people not available to do the work, poor communication, complacency, etc. A categorization of these issues and commonalities allowed initial identification of the attributes of DMS (e.g. agents, activities, infrastructure, knowledge and information) and the organisational aspects which could be affected by the DMS (e.g. internal and external variables, cultures and level of decision-making) and provided the development of the initial framework.
94
E.M. Molloy, M.A. Sinclair & C.E. Siemieniuch
Pilot studies A number of pilot studies were carried out, to aid development of the framework and to develop methodology for the subsequent studies. Two student-based studies were investigated. Both were year-long design projects specifically mimicking organisational and technical aspects of full-scale engineering projects. These were followed by two field studies, one on the design and build of an award-winning innovative educational building, and an engineering study, evaluating a research and engineering centre. All of the studies looked at the identification of key decision points and the mechanisms of decision-making and decision support, leading to further development of the framework (and its associated system). Some key initial findings were: (1) not all decisions are consciously made, creating a rich environment for miscommunication and emergent behaviour; (2) corporate and top level strategy is not always flowed down or instantiated in day to day activities, which can impact on decision rationale; and (3) small scale organisational success with stakeholder involvement does not always scale up and can have unexpected outcomes, meaning the right processes may not work as expected if applied for misguided reasons.
Case studies This research involves two industrial case studies, in addition to the pilot studies outlined above. In both the studies, a qualitative approach to data gathering was used. Preliminary work involved analysis of existing documentation followed by a combination of email communication, interviews and workshops. The studies took place within a multinational aerospace organization experiencing the issues of the product-service transition. Both focused on the critical process of deciding whether or not to bid for a contract, exploring identified problems at different levels of activity decomposition. Data were gathered on activities and roles, information flows and decision points, along with process and problem perception and other contextual details. An outline of these studies is included in the table below.
The DMS framework A representation is given in Figure 1. The row headings are component parts of a DMS. The column headings are aspects of the overall system or system of systems, which could affect or be affected by the DMS. These are outlined below.
Components of the DMS The following are the constituent parts of a DMS: • Agents: DM (decision-making) agents may be human or software based, for example, who are involved in the DM process. • Activities: DM activities are those activities necessary for a decision to be made.
Decision-making after the product-service shift and some implications
Table 1.
95
Outline of industrial case studies that helped to develop the framework.
Study
Problem statement
Overview of key findings
Industrial study 1
A stage of the bid process had become congested with mandated activities and reviews, leaving little time to complete all necessary work to progress the bid. Bids could be small, filling niches, or billion-dollar, ‘bet the company’ size. The study explored whether all the activities were necessary.
Industry study 2
There was concern about the quality and consistency of a particular pre-bid technical review. There seemed to be a lack of understanding of the training requirements for those involved in the review; one obvious knock-on effect being an incorrect assessment of the technical issues in bids. This could be very unfortunate in big bids.
• A change of focus has occurred with contracts written on service-based ideas with a more delicate balance of capability and profit. For the bid process, this represents new risk and its analysis is not as mature as it could be to deal with this. • The change has encouraged overlap in review activities between different functions • It is not enough to adapt existing processes to deal with these new contracts – new process must be developed and employed • Not all decisions are consciously made, especially during status review. • For the new service-based contracts there was uncertainty about what should be reviewed, with concern that some things are not reviewed at all, leading to unknown risk. • Purpose and importance of activities was not clear, which has issues for bid tailoring and consistency of output. • These things need to be addressed before appropriate training can be developed.
• Infrastructure and technology: That which provides support for the DM process. • Knowledge and information: DM knowledge and information is that which is necessary for a decision to be made. The DM knowledge and information flows through the infrastructure and technology, around the DM activities to the agents to allow decisions to be made.
Impacting variables on the DMS These include: • • • •
Internal variables: Contextual issues (e.g. stage of lifecycle) Environmental variables: External influences (e.g. legislation, partners) Organisational variables: Structure, empowerment, risk – taking, etc. Level of DM: Strategic, tactical or operational.
96
E.M. Molloy, M.A. Sinclair & C.E. Siemieniuch
Figure 1. Top-level snapshot of the framework, illustrating a DMS audit. Numbered circles are issues. For example, issue 2 is the lack of personnel in charge of specific parts of the process. Linked circles show multiple categorizations of the same issue. Note that some combinations of void cells can imply that the audit was insufficient.
Using the DMS framework The aim of the DMS is to assist the audit of any DMS within an organization and to aid its dynamic reconfiguration for the extended lifecycles of the product-service shift. Figure 1 is a top-level view of the prototype DMS Framework. Issues identified in an audit may be oriented in this framework, according to the component of the DMS and impacting variable concerned. Mapping of issues allows ‘quickview’ comparisons of both clusters and of voids to highlight areas for investigation. This surfaces issues and allows stakeholders to commence discussions to enact improvements; it is essentially a tool to initiate and direct discussion. It does not present ready-made solutions. Work is in progress on the application and use of the framework and its databases and techniques. Detailed use scenarios are being developed. It is currently envisaged that will be possible to apply the tool both for prospective and retrospective work. On the negative side, there are some issues that the framework does not currently address. These include training and workload. However, as noted above, the framework has been used to investigate problems in these areas. Furthermore, the use of the DMS framework and associated processes relies heavily on honest identification of issues and subsequent categorization. Given that few people are able to understand more than a small part of their own organization, and given the power
Decision-making after the product-service shift and some implications
97
relationships and politics to be found in any organization, these are not easy goals to achieve. Finally, an upgrading process will be made available for the framework, to allow it to be tailored for a given organization and its ever-changing context and to provide updates as new information becomes available.
Validation A final case study was used as validation for this work. The aim was to validate both the tool and the process for its use. The study took place in a manufacturing organisation that provided promotional products for retailers, a very different industry to that which formed the focus of the primary studies for this work. During the time of the study, the organisation was transitioning to lean manufacturing techniques after a commercial ‘Near Death Experience’ ((OSTEMS) 1998). The problems they were encountering included: • Management of a transient workforce • Loss of knowledge through loss of key personnel • Issues arising from changing customer requirements. The DMS framework was used alongside more established techniques (such as the Role Matrix Technique (Callan, Siemieniuch et al., 2005) and IDEF activity modelling ((IDEF) 1993) and has been shown to complement such techniques, providing extra contextual information from a decision making point of view and helping to identify some potential route causes for issues ascertained elsewhere. The tool was validated insofar as: • It was useful for the company • It was able to reveal a number of relevant issues that had not been considered The study also highlighted the expert level of knowledge required to use the prototype version of the tool. A more usable form will need to be developed before widespread use.
Commentary The framework does not provide ‘right’answers, but informs organisations about the characteristics of their DMS to enable them to be resilient in unexpected situations. With new contract and project types, there is a requirement for new processes, which may require more (different) people and resources and will require wider knowledge. These processes must be accepted and integrated into the organisational processes if they are to be effective. Considerations from agility show that a paradigm shift is required. This includes the hard bits of the organization to change – culture, trust, leadership etc.
98
E.M. Molloy, M.A. Sinclair & C.E. Siemieniuch
There will also need to be a significant upgrading of the infrastructure that joins the community of organizations and people that deliver and use the capability. This must at least accompany, if not be in the lead, of the changes outlined above. With the growth of ‘through-life capability’contracts comes a shift in responsibility. Suppliers’ responsibility to provide a through-life service means the decisions they make are now long-term. This calls for better decision support, with better identification and assessment of risk (technical and business). These form part of the new processes and must be easy to use, if not implicit within regular working practices. If a decision is not consciously made, the outcome is neither known nor accounted for. This can lead to unanticipated behaviour (emergence), increasing risks and missed opportunities. With this interlinking of issues, a multi-disciplinary approach is a necessity in any such contract. The collaboration requirements mean that whole supply chains must work in partnership. The multi-disciplinary approach will cross organisational boundaries, calling for a better alignment between different organisational processes, as noted in the industrial studies. Without full information flows, trust and partnerships are essential. By focusing attention away from the facility at the point of handover to the services supported by that facility over the longer term, the current changeover to product-service projects provides new challenges for decision-making. With service typically being consumed or used as it is produced, it is probable that each instantiation will be different. It becomes fundamental for a service organization to ensure that it is resilient enough to cope with a very changeable environment. All of the above emphasises the key message of this paper: effective and appropriate, embedded decision support is essential for successful service organizations, and tools for this task must be flexible enough to withstand the manifold challenges of complexity. This framework tool is intended to help organisations to achieve effective and appropriate, embedded decision support.
Acknowledgements The work presented herein was undertaken in the Knowledge and Information Management (KIM) Through-Life Grand Challenge Project (www.kimproject.org) funded primarily by the Engineering and Physical Sciences Research Council (EPSRC Grant Number EP/C534220/1), the Economic and Social Research Council (ESRC Grant Number RES-331-27-0006.
References (IDEF) (1993). Integration Definition For Function Modeling (IDEF0). Gaithersburg, MD 20899-3460, National Institute of Standards & Technology. (OSTEMS), A. (1998). Understanding Best Practice for key elements of the New Product Introduction Process, DTI/ESRC Business Processes Resource Centre, University of Warwick, Coventry, UK.
Decision-making after the product-service shift and some implications
99
Anon (2008). Implementing Systems Engineering in Defence. D. S. Engineering, UK Ministry of Defence. Callan, K., C. E. Siemieniuch, et al. (2005). A case study example of the role matrix technique. INCOSE 2005, Rochester NY. Coakes, E. and Clarke, S. (2006) Encyclopaedia of communities of practice in information and knowledge management, London, Idea Group. Finlay, P. (1989) Introducing decision support systems, Oxford, NCC Blackwell. Gregg, D. (1996). Emerging challenges in business and manufacturing decision support. The Science of Business Process Analysis, ESRC Business Process Resource Centre, University of Warwick, Coventry, UK, ESRC Business Process Resource Centre. MOD (2005). Defence Industrial Strategy. M. o. defence, Her Majesty’s Stationery Office: 143. Molloy, E.M., Siemieniuch, C.E. and Sinclair, M.A. (2008) “A new decisionmaking systems framework”, in Contemporary Ergonomics 2008, Bust, P. (ed), Taylor & Francis: 55–60. Oliva, R. and R. Kallenberg (2003). “Managing the transtion from products to services.” International Journal of Service Industry Management 14(2): 160–172. Rasmussen, J. (1997). “Risk management in a dynamic society: a modelling problem.” Safety Science 27(2/3): 183–213. Rasmussen, J. (2000). “Ergonomics.” Ergonomics 43(7): 869–879. Rycroft, R. W. and D. E. Kash (1999). The complexity challenge. London, Pinter. Silver, M.S. (1991) Systems that support decision makers: description and analysis, Chichester, John Wiley & Sons. Simon, H. A. (1997). “The future of information systems.” Annals of Operations Research, 71: 3–14. Vaill, P. B. (1998). The unspeakable texture of process wisdom. Organizational wisdom and executive ocurage. S. Srivastva and D. L. Cooperrider. San Francisco, Jossey-Bass: 25–39. Wenger, E. (2000), “Communities of practice and social learning systems”, Organisation articles, 7(2): 225–246.
GOVERNANCE, AGILITY AND WISDOM IN THE CAPABILITY PARADIGM M.A. Sinclair, M.J.D. Henshaw, R.A. Haslam, C.E. Siemieniuch, J.L. Evans & E.M. Molloy Loughborough University BAE SYSTEMS As a consequence of an investigation into Engineering Governance, an exploration of Agility within the realm of national defence, and the burgeoning understanding of Complexity within systems-ofsystems, the usual conclusion is supported: it’s the people that are the most important asset. The paper discusses some of the structural aspects for ensuring Through-Life Capability Management in changing environments, and some tools to enable managers and others to plan these structures
Introduction The paper presents an argument based on the assimilation of a number of different projects, all carried out in manufacturing industry, covering engineering governance (Nendick, Hassan et al., 2006; Nendick, Siemieniuch et al., 2006), supply chains (Siemieniuch and Sinclair 1999; Siemieniuch and Sinclair 2000; Siemieniuch and Sinclair 2002), knowledge management (Siemieniuch and Sinclair 1999; Siemieniuch and Sinclair 2004), and organisational preservation (Siemieniuch and Sinclair 2002; Siemieniuch and Sinclair 2003; Siemieniuch and Sinclair 2004), and decision support (Molloy, Siemieniuch et al., 2007; Molloy, Siemieniuch et al., 2008). Aspects of these have been assimilated into a discussion of the effects of complexity, the growth of systems-of-systems, and the role of Human Factors in their long-term survivability (Sinclair 2007). This paper extends this assimilation to include governance, agility, and the importance of wisdom in organisations.
The scenario The current shift in the manufacturing paradigm for the capital goods industry away from selling a product and towards a longer-term capability that delivers an effect (“we no longer sell you computers at a one-off cost; we sell you an fully-serviced IT infrastructure, for an annual fee, that frees you to concentrate on your business.”), necessitates a significant shift in the way that supplier companies organise their operations. The scenario is shown in Fig 1; a supply chain to an end-customer (e.g. providing jet engines to an airline, on a ‘power by the hour’ basis, so that the airline can rely on a given level of readiness in its fleet, and has no need to resource and manage this. For both the sides this results in a smooth, predictable, long-term cash flow). Azarenko, Roy et al. (2008) provides an interesting discussion of this approach. 100
Governance, agility and wisdom in the capability paradigm
101
Figure 1. Illustration of a supply chain. At right is a customer, utilising the assets provided by the suppliers. At left is the constellation of suppliers supporting these assets through life. OEM = Original Eqpt Manufacturer. Cap Gp = a capability grouping of assets to achieve an effect. For the companies in the supply chain, there is a requirement to extend the operational envelope; it is not enough to design and deliver the product to the customer’s specification; now, the supply chain must anticipate the usage of the product and the support it will require, in conjunction with other products, in a changing environment, for a long time ahead, and reorganise itself accordingly. Hence, as Ashby’s Law indicates (Ashby 1956), the supply chain must become more complex to handle this increased operational complexity. There is also an increased requirement of information transfer; unfortunately, for reasons of security and commercial confidentiality, while data transfer may increase significantly, information and knowledge transfer rates may not. Considered as a multi-agent environment, Fig 1 has the following characteristics (Gregg 1996): • An evolving, uncertain, environment (client, suppliers, etc.) • Many agents, of different kinds, with lots of connections • Communicating in parallel • Evolving agents, with many steady states • Interactions between agents across organisational boundaries • Interactions between different goals within an agent • Interactions between agents with different goals • Language/culture differences • Restricted time (deadlines, interruptions, etc.) As Gregg pointed out, if just a few of these are present, then the system’s behaviour is no longer predictable; emergent, probably unwanted, behaviour is possible. It is with governance that one may ameliorate this emergence.
Engineering Governance Governance is concerned with three questions: • Are we doing the right things? • Are we doing those things right? • How do we know this?
102
M.A. Sinclair et al.
A top-level view of engineering governance covers four perspectives: • • • •
Enterprise – risk management (‘ensure an appropriate market presence’) Performance – delivery of capability for business benefit (‘make a profit’) Asset quality – delivery to specification (‘get paid’) Legal/social – meet legal/social requirements (‘stay out of jail, and be respected’)
Humphrey (1989) said it well: “. . . a disciplined environment empowers the intellect, whereas the regimented environment marginalises it.”, and this should be the goal, while meeting the top level perspectives. It requires both the freedom for people to exercise their capabilitiers whilst fulfilling the needs of the organisation; discipline implies freedom within constraint, and the organisation, while creating an environment in which to prosper, should also address the issues of disciplinary transgression; there must be a justice system provided as well as a reward system to complement the organisation’s processes (Reason 2001). The processes themselves require documentation and ownership to ensure their long-term integrity; bearing in mind Humphrey’s encomium above, it becomes evident that the principles and precepts of socio-technical theory are applicable here (Cherns 1976; Davis 1982; Eason 1988). Tools exist to map processes (e.g. IDEF0, UML/, SysML, RADS); others to map both role relationships and cultural characteristics in relation to processes are in somewhat shorter supply, examples being the Role Matrix Technique (Callan, Siemieniuch et al., 2006), Social Network Analysis and the Cultural Values Modelling Tool (Siemieniuch and Sinclair 2006; Hodgson and Siemieniuch 2008), FOCUS’ and the Competing Values Framework. Because of the long-term nature of capability provision, governance must necessarily include knowledge lifecycle management and its propagation within a regime of constant personnel turnover (Siemieniuch and Sinclair 2007). A particular issue in governance is that when audits, reviews, and other governance assessments are carried out, with whatever metrics are deemed appropriate at the time, it is tempting to compare against benchmarks arising from past performance. Unfortunately, past performance is not a good guide to future demands; it is appropriate to discuss agility at this point before returning to governance, the reason being that the demands of agility may render past benchmarks irrelevant. One particular aspect of agility which demands good governance is the fact that if an organisations would gain in agility, there is a pre-requisite that it be wellgoverned. This is because any permanent gain in agility must stem from some degree of re-organisation of the enterprise.
Organisational agility It is in the speed of response and the adequacy of the response to altered circumstances (whether planned or unplanned) where competitive advantage may lie; in the commercial world, one might consider the travails of Microsoft in producing the Vista operating system compared to Apple which produced two operating systems and also changed platforms in the same time frame. Organisational agility arises from both robustness and resilience. Furthermore, there are different classes of agility, so some clarity of terms is required at this
Governance, agility and wisdom in the capability paradigm
103
point. Organisational robustness is the result of pre-planning; risk analysis followed by risk amelioration in the design, development and subsequent upgrading of the capability system that is instantiated. It ensures that we have available the necessary assets that comprise the capability together with the plans to redeploy these with minimum effort and adjustment to address the risks we have foreseen, thus being able to maintain organizational and system integrity. Organisational resilience includes the capability to adjust to unexpected events as well. As above, we assume that system integrity will be maintained, but it will be an adapted form of integrity. The implication is that our capability assets will be rearranged in real time into new configurations, and some will be used in novel ways, to deliver the goals. Resilience represents second- and third-loop control, with a consequent requirement for wisdom, knowledge and wide-ranging, timely information for its success. Organisational agility addresses both the extent of change and the rate of change to the new circumstances pertaining in the organisation’s context. Necessarily, this includes support for delivered capability as well. In this respect, a thoughtful paper from a defence perspective to be presented at an INCOSE conference discusses four levels of agility with regard to time as shown below (Mackley 2008): • • • •
Tactical Operational Strategic Lifecycle
respond within a day or so respond between a week and a year or two respond within several years respond over a longer period.
On a Tactical timescale one adapts with what one has, essentially by re-allocating resources and changing the patterns by which they interact. These changes may be robust, because they were foreseen and alternative plans for reconfiguration exist, or they may be ad hoc, because of an emergent surprise. As one goes down the scale, greater change is possible, but more slowly, and at greater cost (i.e. we can demonstrate limited agility very quickly, but more extensive agility beyond this will only happen with more time). This is because the assets themselves can now be changed, as can their interaction capabilities – we can adapt, alter, replace, extend, innovate, etc. This now involves the supply chain, reaching back a distance both in organisation, co-ordination and in time. Foresight is the key issue here; if the supply chain that delivers the assets has looked ahead sufficiently acutely, then the assets will have most if not all the resilience built in (i.e. we will have ensured robustness), allowing us to be Tactical, rather than Operational or Strategic in our response. Hindsight has to do with the reactivity of the supply chain given the emergence of a problem; again there is the opportunity of fast reaction, but the organisations involved must be geared towards this and must have long-term access to relevant information and understanding – from the suppliers’ perspective they need ‘reach-forward’ fairly extensively into the customers’ information environment. And, equally importantly, there must be a considerable level of trust among the suppliers involved in rearranging any assets. Clearly, governance must address these issues of change as well; agility is a never-ending exercise. It is also evident that wisdom has an important part in this,
104
M.A. Sinclair et al.
particularly where resilience is required. As indicated above, resilience implies something of a step into the unknown; we must understand the situation which is developing in front of us, and must respond well and quickly to it.
The relevance of organisational wisdom The argument developed to this point is that the shift to a service paradigm for capability introduces greater complexity into the supply chain, requiring a more complex structure to address it. Furthermore, the long-term nature of the capability shift means both that the more complex structure will have to adapt to changing circumstances, and to required increments in agility, with continuous, perhaps far-reaching effects of knowledge loss due to personnel turnover. While good governance will ameliorate the likelihood and effects of emergent behaviour, and knowledge management techniques will overcome most of the knowledge loss issues, there is a role for wisdom in economising on the effort to govern and to manage knowledge; uniquely, wisdom also addresses effectively the ‘unknown unknowns’ to which a capability service is exposed. Wisdom thus provides a third strategy for organisational benefit, and we address this now. It is somewhat staggering to discover that organisational wisdom is a topic that has been virtually ignored for over a century by the soft sciences (Ardelt 2005). While there has been recent interest in the wisdom of individuals and social groups, there appears to be very few documents which discuss wisdom in organisations (Hammer. 2002 does this very well). Wisdom combines an ethical perpective with perception, knowledge, experience and communication. Because experience implies the passage of time, wisdom tends to be associated with age, but the link is not strong – and could be made weaker by the use of simulation and gaming techniques. Some characteristics of wisdom: • Wisdom is acknowledged; it cannot be claimed • Self-serving behaviour is never considered wise. Ethical behaviour usually is. • Wise people receive the same inputs as the rest of us; what distinguishes them is this combination: they see bigger patterns in these inputs, seek more rich corroboration, understand better the alternative decisions and their outcomes, and deliver solutions which are better at that time. • People may be ‘wise’ in regard to some aspects of the environment, and ordinary in relation to other aspects. Hence, ‘wisdom’ should include ‘in relation to X’. • Wisdom is not free of errors; in a changing world, wise performance at one time may be silly, irrelevant or wrong at another. • Wisdom is usually in short supply. Hence, it needs management. Industry’s typical perspective , as explicated by Hammer, is as follows: • It’s needed at the top. • Identify your talented people; create a career path to give them experience (and make them wise).
Governance, agility and wisdom in the capability paradigm
105
• Improve the rest; wisdom works best within a knowledgeable environment. • Create a mentoring function, with those at or near retirement as the mentors. • Retain your wise people. However, in relation to Fig 1, and coupled with the issues of governance and agility outlined above, arguably this is not enough. The orthodox approach is positive, but not focussed on actual needs, processes and the changing environment; we need wisdom at the top of the organisation – but where else as well? The capability paradigm implies that wisdom will need to be distributed horizontally as well as vertically, and especially where the capability is delivered and realised. In turn, this implies that career paths may have to be lateral instead of always-upwards; there may need to be ‘gates’ such as ‘You cannot be appointed to this position until you have performed X; you cannot perform X until you have experienced Y. . .’, with governance metrics, reward systems and a culture that appreciates the worth of wisdom. If this argument is accepted, then it follows that the Human Resource Management function in organisations has a much larger role to perform. We offer Fig. 2 as
Figure 2. A roadmap to illustrate progress towards a wise organisation.
106
M.A. Sinclair et al.
a roadmap for further progress, for organisations to achieve ‘fitness for functions’ (Jenks 2004; Dixon, Brown et al. 2007).
Summary The paper has outlined some of the consequences of a shift to a capability perspective for the capital goods marketplace. Most of the issues stem from the problems of complexity, allied to the long-term nature of capability support. It is argued that governance is a key issue in ameliorating the effects of complexity, but that governance is not a simple issue, given the organisations must consider agility as well. Furthermore, wisdom is at least as important as governance is ensuring that organisations are fit for purpose in a capability environment. It is the people that matter in this, and their integration with the technology is a key issue.
References Ardelt, M. (2005). Foreword. A handbook of wisdom. R. J. Sternberg and J. Jordan. New Your, Cambridge University Press: xi–xvii. Ashby, W. R. (1956). An introduction to cybernetics. London, Chapman & Hall. Azarenko, A., R. Roy, et al. (2008). “Technical product-service systems: some implications of innovation within the Machine Tool industry” Journal of Manufacturing Technology Management in press. Callan, K., C. E. Siemieniuch, et al. (2006). “A case study example of the Role Matrix Technique.” International Journal of Project Management 24(6): 506–515. Cherns, A. B. (1976). “Principles of socio-technical design.” Human Relations 29: 783–792. Davis, L. E. (1982). Organisational Design. Handbook of Human Factors. G. Salvendy. New York, Wiley: 433–452. Dixon, K., S. Brown, et al. (2007). Integrating the intelligent enterprise. 17th Annual Symposium of the International Council on Systems Engineering, San Diego. Eason, K. D. (1988). Information technology and organisational change. London, Taylor & Francis. Gregg, D. (1996). Emerging challenges in business and manufacturing decision support. The Science of Business Process Analysis, ESRC Business Process Resource Centre, University of Warwick, Coventry, UK, ESRC Business Process Resource Centre. Hammer, M. (2002). The Getting and Keeping Of Wisdom – Inter-Generational Knowledge Transfer in a Changing Public Service. Ottawa, Research Directorate, Public Service Commission of Canada. Hodgson, A. and C. E. Siemieniuch (2008). A software tool for evaluating the effects of culture on military operations. AACC-08: Adaptive Agents in Cultural Contexts. Arlington, VA, American Association for Artificial Intelligence.
Governance, agility and wisdom in the capability paradigm
107
Humphrey, W. S. (1989). Managing the software process. Reading, MA, AddisonWesley. Jenks, C. L. (2004). “The well-being of social systems.” Systems Research and Behavioural Science 21: 209–217. Mackley, T. (2008). Concepts of agility in Network Enabled Capability. The NECTISE Conference. Leeds University, UK, BAE Systems. Mackley, T. (2008). The role of lifecycle systems in the through-life engineering of system solutions M. A. Sinclair. Cranfield University School of Engineering: Conference paper yet to be publshed. Molloy, E.-M., C. E. Siemieniuch, et al. (2007). The Effects of Complexity on the Product-Service Shift. International Conference on Complex Systems. Boston, MA. Molloy, E.-M., C. E. Siemieniuch, et al. (2008). Developing a DecisionMaking Systems Analysis Framework. KIM Project Conference. Reading University, UK. Nendick, J. V., M. Hassan, et al. (2006). ‘Good Engineering Governance’ – an issue for Ergonomists. IEA2006: International Ergonomics Association 16th Congress. Maastricht, International Ergonomics Association. Nendick, J. V., C. E. Siemieniuch, et al. (2006). Ergonomic aspects of ‘good engineering governance’ in the design process. Contemporary Ergonomics. P. Bust. London, Taylor & Francis: 62–66. Reason, J. (2001). Managing the risks of organisational accidents, Ashgate. Siemieniuch, C. E. and M. A. Sinclair (1999). “Organisational aspects of knowledge lifecycle management in manufacturing.” International Journal of Human-Computer Studies 51: 517–547. Siemieniuch, C. E. and M. A. Sinclair (2000). “Implications of the supply chain for role definitions in concurrent engineering.” International Journal of Human Factors and Ergonomics in Manufacturing 10(3): 251–272. Siemieniuch, C. E. and M. A. Sinclair (2002). “On complexity, process ownership and organisational learning in manufacturing organisations, from an ergonomics perspective.” Applied Ergonomics 33(5): 449–462. Siemieniuch, C. E. and M. A. Sinclair (2002). The role of the Process Owner in avoiding the ‘drift to disaster’ in processes. 2nd International Conference on Occupational Risk Prevention, Maspalomas, Gran Canaria. Siemieniuch, C. E. and M. A. Sinclair (2003). Changes in organisational roles when disaster strikes, and the lessons that can be learnt from this, in the manufacturing domain. Human factors of decision making in complex systems, Dunblane, Scotland, University of Abertay, Dundee. Siemieniuch, C. E. and M. A. Sinclair (2004). Long-Cycle, Distributed Situation Awareness and the Avoidance of Disasters. Human Performance, Situation Awareness and Automation: Current Research and Trends. D. A. Vincenzi, M. Mouloua and P. A. Hancock. Mahwah, NJ, Lawrence Erlbaum Associates. 1: 103–106. Siemieniuch, C. E. and M. A. Sinclair (2004). “Organisational Readiness for Knowledge Management.” International Journal of Operations & Production Management 24(1): 79–98.
108
M.A. Sinclair et al.
Siemieniuch, C. E. and M. A. Sinclair (2006). Impact of cultural attributes on decision structures and interfaces. 11th ICCRTS Coalition Command and Control in the Networked Era, Cambridge, UK, US DOD. Siemieniuch, C. E. and M. A. Sinclair (2007). “Using corporate governance to enhance ‘long-term situation awareness’ and assist in the avoidance of organisation-induced disasters.” Applied Ergonomics 39(2): 229–240. Sinclair, M. A. (2007). “Ergonomics issues in future systems.” Ergonomics 50(12): 1957–1986.
DESIGN
THE ROLE OF ERGONOMICS IN TECHNOLOGY ORIENTED PROJECTS; THE INTELLIGENT ZERO EMISSIONS VEHICLE PROJECT A. Woodcock & J. Taylor The Design and Ergonomics Applied Research Group, Coventry School of Art and Design, Coventry University, UK Social, political, economic, ecological and technological factors will bring radical changes to future transport systems. The Intelligent Zero Emissions Vehicle (IZEV) project concerned the design and installation of intelligent, wireless computing, and hydrogen fuel cell technology in a taxi and narrow boat. Such telemetry will enable fleet owners, drivers and passengers to pull down real time information pertinent, concerning, for example, route and tourist information, fuel efficiency and location information. This will enable travelers to make informed, intelligent decisions about their travel behaviour. The creation of such demonstrators provides opportunities for stakeholders to understand how technological developments will influence their everyday lives and to provide input future scenarios. Their feedback can help designers better understand requirements, modify their designs and consider ways of presenting their ideas for greater public awareness and acceptance. This paper describes both the end user involvement which occurred in the four month IZEV project and reflects on the opportunities such projects offer for enquiries into ergonomic issues.
Introduction ‘It is widely recognised that the world has no more than twenty years to meet the urgent challenges of climate change and oil depletion.’ Harris (2007). Approximately a third of Britain’s CO2 emissions are from transport. These are arguably the most intractable forms of CO2 emission and certainly the ones rising fastest. The West Midlands faces the problems of many urban conglomerations in the UK and has recently been awarded £26 million to tackle transport related problems. An array of solutions are possible to alleviate traffic related such as congestion charges, toll roads, pedestrianisation, vehicle taxes, speed limits and bumps, and the design of energy efficient vehicles and fuels. The Intelligent Vehicle Demonstrator Programme, funded by Advantage West Midlands under the Science City initiative, sought to address transportation problems by using in-vehicle technology to enhance transportation networks within a City Centre environment. To this end a number of applications were designed and 111
112
A. Woodcock & J. Taylor
installed in a car and narrow boat to support remote tracking, monitoring and fuel usage, as well as ‘pushing’ interactive content to drivers and passengers through a dedicated touch screen. The rationale for delivering more information on vehicle performance, routes and driver behaviour is that the passenger, driver or fleet owner can use this information to make intelligent decisions about their journeys. The immediate questions this gave rise to related to whether the technology was needed, or offered services over and above those already available, how such systems would be used, and whether they had been designed appropriately.
The vehicles The narrow boat Canals are an under utilised resource. ‘One of the most energy efficient means of moving goods is by canal and the threats of global warming and oil depletion are resulting in a resurgence of interest in this means of transportation.’ Harris (2007). The zero-emission hydrogen hybrid narrow boat (Figure 1a) was developed by the University of Birmingham to demonstrate how magnet and fuel cell technologies can be used to power inland waterways craft more efficiently than carbon based alternatives. Green hydrogen is considered a clean fuel as it has a minimum impact on the environment and can reduce the levels of carbon dioxide and other greenhouse gas emissions.
The microcab The Microcab (Figure 1b) was developed in 2001. It has been the subject of investment, iterative design and review by Jostins and others (e.g. Atkinson, 2007). It has evolved into a series of silent, small urban vehicles operated by scooter-like handlebar and lever controls, with zero emissions, suitable for use as a taxi or light freight carrier. It is powered by hydrogen fuel cells from which the only emission is water vapour. Weighing 250 kg, it is an ultra light, urban vehicle with a maximum seed of 30 mph and a range of 100 miles.
Figure 1.
1a) The narrow boat and 1b) Microcab.
The role of ergonomics in technology oriented projects
113
These development vehicles formed the carriers for the intelligent systems, which were designed to track the vehicles use, monitor fuel use and provide information to vehicle occupants. It was expected that the incorporation of such technology will increase the efficiency, effectiveness and enjoyment of vehicle use. The following section provides details of the technology.
The intelligent systems These consisted of four parts 1) Remote, (online) live tracking of vehicles – overlaid onto Microsoft Virtual Earth maps. This enables vehicles to be tracked to street level and their movements displayed in real time (Figure 2), 2) Remote tracking of vehicle journeys, fuel, engine temperatures etc showing how the vehicle is being driven (Figure 3), 3) In-vehicle touch screen to support individual email accounts, web cams, hands free voice over IP Phone calls, GPS, MP4 and internet access, 4) Location based content ‘push’ to vehicle touch screens (to include parking information and what’s on guides) The industrial partners, Shoothill and RDM expect that such systems will be used, for example, by transport firms to monitor the current location and expected arrival times of vehicles, to monitor traffic flows, to plot the quickest and most costeffective routes, and show individual vehicle performance. Given that it is now possible to collect a vast amount of information from vehicle sensors and send this to different users, one of the main challenges involves understanding what information is needed for which user, at what time and how this can be presented. For example, currently, research technicians are interested in gathering fuel and performance information such as stack temperature, voltage, current etc; drivers may be interested in fuel consumption given the shortage of re-fuelling stations. To explore this, a portal was developed (see Figure 3) which
Figure 2.
Snap shot of vehicle location and journey details.
114
Figure 3.
A. Woodcock & J. Taylor
Representation of system showing real time route and fuel usage.
displays real time information. The portal developed (Figures 2 and 3) provides a way of communicating the potential of new technology to ‘non-technical’ visitors in innovative ways, by showcasing car routes and a live camera feed from the car.
The role of ergonomics Similar to many funded projects, IZEV was technology led. Its main objective was to produce a series of demonstrators, exemplifying how technology could be used to enhance transport systems of the future. In such projects there may be little scope for ergonomic investigations – the time frame is short (in this case 4 months), user trials are dependent on technology being installed and tested prior to the end of the project, and many of the underlying issues, such as those relating to interface design and task interactions have been ‘addressed’ by the developers. Lessons learnt from this and previous projects (e.g. Woodcock and Scrivener, 1997) indicate that detailed user trials (as opposed to those which occur in prototype development) can only occur in a follow on project, with funding organized to ensure that this is not simply a requirement but is a fundamental part of exploitation and commercialization. As far as possible, studies were undertaken to understand the opportunities the new technology would provide and how it would be received. Additionally two technology trials were conducted with early versions of the systems in both the narrow boat and the Microcab to discover usability and acceptance issues. These are both related in the following section. However, because such systems are only ‘working prototypes’, which suffer because of interoperability problems and limited functionality, potential users are often disappointed with their experiences.
The role of ergonomics in technology oriented projects
115
This is problematic. We need to involve end users in usability testing and discussing the wider implications of planned technology usage as early as possible. However, the systems they experience at such early stages may be far removed from what the software developers are planning or imagine they are showing. This may bias and frustrate users and reduce opportunities for the full discussion such systems require. The authors firmly believe that part of their responsibility in such projects is about increasing public awareness and providing opportunities for public engagement. In the IZEV project, as we were not able to use, test or demonstrate fully working prototypes, we developed persona and usage scenarios to understand how the technology might be used. Mindful of the need to raise greater awareness of the underlying rationale and technology, these were developed with ProMedia, along with more technical material, into ‘an educational’ DVD for councils, teachers, end users, wider stakeholders and future investors to understand the potential of the research.
User groups and usage of intelligent, zero emission vehicles Through a state of the art review, qualitative investigations with project partners, interviews with members of the public and different stakeholders, issues relating to the user groups and usage of the technology were discovered. It was predicted that there could be three main user groups for the small road vehicles (whether these acted as a light van or a Microcab) – the passenger, driver and fleet owner. Three main user groups were distinguished for the narrow boat – the freight or fleet company, owner occupier and holiday maker. Persona were developed for each of these groups. 25 interviews were conducted with taxi drivers in Coventry in order to understand the way they work, their experience of and attitudes towards technology. Current systems provide taxi drivers with satellite-navigation, radio, meter and radio. Surprisingly, not all taxis had a meter and only a few had satellite-navigation. All drivers were happy with their current systems, the information which was provided and the way it was presented. The radios were problematic at busy times when it was difficult to interact with the controller. The fleet owners similarly expressed satisfaction with current technology. Concerns were expressed over touch screens (and to a certain extent satellitenavigation) as these could lead to vandalism and theft. Drivers already felt unsafe at night when picking up fares from in and around the city centre, as passengers are often drunk and show antisocial behaviour. Only three drivers had had cc-tv installed in their cabs as part of a pilot system to address driver safety. They would not contemplate installing further systems until security issues had been resolved. Additionally drivers were worried about taking their eyes off the road when they used such systems To summarise, 14 of the drivers (just over half) were interested in the system, 6 would not consider it because of the security issues, and the remaining 5 felt they were not computer literate enough to use the described system and felt it would
116
A. Woodcock & J. Taylor
complicate their job instead of making it easier. For the drivers that said they would consider one, they would like: 1. Whilst stationary – some form of entertainment (mostly internet), details of points of interest such as the location of the nearest petrol station. 2. Whilst going to pick up a passenger – Drivers would like to be able to use GPS to show directions to the fare, to use the system to communicate with the office to accept jobs and get addresses. 3. Whilst driving to their destination – Most drivers would like route information and to be able to calculate and charge the passenger for the journey. 4. Whilst returning to base – Use the system to look for more jobs near their current location. Approximately 50% of the drivers said that they did not record information about their fuel consumption, either because they were not interested in this, did not understand the concept or because it was not yet mandatory. 12 drivers recorded information to work out the cost of fuel in comparison to their earnings. Vehicle emission rates are inspected once a year by the Council. If a vehicle has too high a level it is taken off the road until its emissions are within regulations. This can lead to loss of earnings. Therefore, if drivers had a way of knowing this in advance, they would be interested in monitoring their emissions. In summary, from discussions and questionnaires with taxi drivers and passengers, found the following barriers to use: • For taxi cab owners – cost of the system, replication of facilities already on the market (e.g. way finding, cc-tv), over complication and security were major issues. Additionally, the drivers were not overly concerned with reducing their vehicle emissions as a way of controlling pollution. • For taxi passengers (who would be able to use services in the vehicles) – payment of services, replication personal facilities (eg phone and computing), the need for certain types of information on a short journey were barriers to use. It was questionable whether passengers on short journeys around town would make use of the services provided. • If the Microcab was going to be used as a light van – major concerns included the availability of fuel and safety issues related to the silent running of the vehicle (also see Hinchliffe 2008). However, fleets of such vehicles will be welcomed by councils and organizations wishing to reduce their carbon footprint. It may be concluded from this that the ‘taxi industry ‘needs to be persuaded of the benefits of such systems. One way of doing this will be to provide fully working demonstrators, however these may ‘skip over’ the issues relating to usability and cognitive overload which need to be addressed through laboratory based studies.
Barge owners A questionnaire was sent to barge owners (19 responses) and comments received from a web forum relating to the use of the proposed technology on barges. The
The role of ergonomics in technology oriented projects
117
responses indicated a need for up to date information before and during the journey relating to locks, turning points, mooring stations, distances, boat yards, stoppages, winding and refuelling points and information which would help in journey planning. The route planner should provide the opportunity to plan the route on how far you wished to travel and provide a list of possible points of interest. It should also show the time, distance, fuel consumption and the number of obstructions (locks etc). Some barge users already use radios and wireless internet to gain updates on conditions and waterways that could affect their journey such as weather reports, engineering points, hazards and flood warnings. They might accept additional technology if it provided them with useful, usable information. Additionally respondents felt that ‘push technology’ could be used to provide information regarding local amenities such as shops, restaurants, waste disposal, wi-fi and communication points, public transport, roads adjacent to or near to the canal, emergency services such as medical support, veterinary services and police. Canal boats pass through, or are moored in high risk vandal areas, any form of route tracking or surveillance would therefore be reassuring to owners and travelers. In terms of on board performance, whether the barge ran on low emission fuel or not, respondents were interested in receiving information on how the boat was running such as engine performance, engine revs, water temperature, oil pressure, battery capacity, speed and currents. Although the responses indicated a level of support for the technology, some negativity was expressed. This fell in to three categories; • worries that it was merely replacing systems already in place, e.g. “What would be available that isn’t already available via Nicholson’s and Google maps?”,“My laptop is a GPS, depth sounder, VHF, weather tracker and AIS receiver”, • cost of such systems in relation to information that already freely (or cheaply) available and meets needs), “There’s already a lot of information available in printed and on-line form”, • lastly, as predicted, some barge owners were anti technology, “Personally I like to get away from computers when I am on the water.” Additionally respondents were split about having the information they required sent to their phones or on an onboard computer system. Some respondents would like information sent to their phones as these were portable and easy to access while on the move. Others would like to check progress and updates on an onboard computer system. The respondents who were negative, felt that the information was already available on internet sites and through books and did not want expensive equipment on their boats, commenting that if a book got wet it would not be ruined or expensive to replace but that a computer or mobile would be expensive and easier to damage or break while on a canal. If a system is going to be made available to barge users it needs to be carefully tailored to the usage context and specifically provide the functionality that is needed. This might be worth investment as two additional markets were found during the research. Firstly, the holiday maker – who would be interested in novelty, points of
118
A. Woodcock & J. Taylor
interest and have the time to read tourist information about entertainment facilities in towns as well as details about the journey. Although this type of information will also be made available in taxis, it may not be used owing to the time of the journey. Secondly fleet owners (not just of holiday boats), but those who are interested in using canals to transport freight. Remote sensor systems could monitor fuel usage, security and progress.
Technology trials Although full technology trials were planned using working prototypes in both the narrow boat and the Microcab. The required systems were not sufficiently integrated with the vehicles to enable these to take place. Instead two preliminary technology trials were conducted on both vehicles using a subset of system functionality. 10 participants aged between 20–64 years evaluated the integrated driver’s systems in the taxi. A task analysis was undertaken to ensure that a representative range of tasks were selected. Anthropometrics was also considered as the system would be placed in the drivers work space envelope. Participants were video recorded using the system in a stationary Microcab, they were encouraged to verbalise as they completed the tasks and completed a post test questionnaire relating to ease of use of the technology. The following findings emerged: 1. In relation to anthropometrics, users had to stretch too far to use the system which was attached to the windscreen. When these results are compared with anthropometric data for the average forward grip reach of British males and females, the screen was found to be positioned too far away to be used comfortably from the normal driving position by even a 50 percentile population. All the participants either had to stretch to use the system or sit forward in their seats. Operation was obstructed by the switch to turn on the window wipers which was knocked several times when reaching past the steering wheel to use the screen, resulting in turning on the wipers. 2. Although the interface did not consist of novel features, and the ‘drivers’ had no other tasks to attend to, they found the interface confusing when selecting postcodes, areas and web pages 3. 50% of participants commented on the small size of the screen and the difficulties this caused for navigation and key pad use. Participants suggested a bigger screen to help navigation and to making internet access worthwhile. 4. Information loading speed was unacceptably slow 5. Most participants were concerned about operating the system whilst driving as they would have to reach over the steering wheel and take their eyes off the road to complete the tasks. They would have provided voice activation and instruction 6. The advanced prediction features of the GPS system caused problems in relation to automatic letter and number predictions for the postcodes and data acceptance. All but one participant saw value in the integrated system and would use one if it was available – although this would depend on cost and the length of time spent on the
The role of ergonomics in technology oriented projects
119
road. The other participant commented “I do not think it was anything particularly new”. Additionally one participant felt that such a system would invite theft.
The technology in the narrow boat As the system installed in the boat had not, at the time of the trials been customized to a water based transport system, only the researchers were invited to provide their comments during a system walkthrough. Although they were enthusiastic about the potential of such a system, most of their comments related to the lack of functionality that specifically met their needs and the fact that the system was not programmed for travel on waterways.
Future potential The Intelligent Zero Emissions Vehicle project linked together hydrogen fuelled vehicles with intelligent systems in close to market exemplars. Such ‘Studies on the performance of the boat {and the car} will establish the viability of hydrogen for energy storage and as a fuel. We wanted to improve the science and engineering in this field by creating a real working example of this type of transport application and to enhance the public’s understanding and acceptance of hydrogen.’ Harris (2007). Taken together the technology provided represents what is feasible in the near future, with suitable levels of investment. From an ergonomics perspective the questions which still need to be resolved include whether the users of transport systems want or will need such systems in the future. Some respondents were ‘anti-technology’; others felt that they are not ‘clever enough’to understand the systems, were content with their current provision and did not see the benefits of integration. Although cost was mentioned, technology is more affordable, and is not so much a barrier. Importantly people are happy with current systems and view integration as adding complexity. However it is likely that if they have mastered and adopted current technology they may move seamlessly to the next generation of devices. Ubiquitous systems are threatening. The ‘big brother’ complex was not raised as much as anticipated – in fact those worried about crime wanted more surveillance, not less. However, this might have been overlooked because very few of the respondents were exposed to the full working system – they did not appreciate that every detail of one’s journey and transport behaviour could be monitored, or what might be done with this information. For respondents, the more complex, or powerful and visible a system, the more likely it was to attract criminal interest. Security measures therefore need to be highlighted in the design to deter would-be thieves and vandals and reassure purchasers. Lastly, even the limited technology trials revealed standard usability problems such as design of feedback and controls, positioning of equipment. In the rush to expand technological horizons, basic usability issues can be neglected. This means that the results and systems developed from such projects must be subjected to a full battery of user testing prior to launch.
120
A. Woodcock & J. Taylor
Summary The Intelligent Zero Emissions Vehicle project has demonstrated hydrogen and wireless technologies in vehicles based at the University of Birmingham for intelligent vehicle management and emissions free operation. Wireless technology may be used to offer an enhanced user experience (furnished through a touch screen monitor) and remote tracking online which supports real time monitoring. The development of such vehicles will enable (for example): • Fleet managers to track vehicles remotely (view location, routes, petrol usage etc.), • Allow parents to ‘live’ track online when the school bus will arrive outside the house or to view the location of the nearest taxi through any internet connection, • Provide wireless cc-TV for taxi drivers, • Deliver free vehicle based internet phone calls within a city centre environment, • ‘Push’ interactive content to vehicles (either to support drivers with traffic and parking information or enhance a passenger’s journey through tourist and what’s on information), • Support local authorities in tracking traffic flows. The inclusion of ergonomists in such projects is most welcome. Ergonomists should be providing advice on the requirements, design and evaluation of future systems, especially ones which will increase the amount of information and tasks people have to manage. However, there has to be real engagement with ergonomists, representative end users and stakeholders at both earlier and later stages than were possible in this project. Time and resources should be allowed for this at the front end of projects. This needs to include full and meaningful dialogue with developers and service providers. Additionally the role of ergonomics needs to be expanded to provide opportunities for stakeholders to consider the wider societal costs and benefits, and usage contexts of such systems.
Acknowledgements The project was sponsored by Advantage West Midlands Science City and was a collaboration between Coventry University and the University of Birmingham, in conjunction with industrial partners, RDM, BT PLC, Shoothill Ltd and ProMedia.
References Atkinson, P. 2006, Can a small taxi be accessible? Notes on the development of the Microcab. In P. Bust (ed) Contemporary Ergonomics, 475–482, Taylor and Francis: London. Jostins, J. 2008, Micro:cab, accessed 29/08/08 at http://www.microcab.co.uk/ index.htm
The role of ergonomics in technology oriented projects
121
Harris, R. 2007, Hydrogen Hybrid Canal Boat: Clean and Silent Propulsion for the Inland Waterways accessed on 29/08/08 at http://www.newscentre.bham.ac.uk/ press/2007/09/Hydrogen_Barge_Launch_21_9_07.shtml Hinchliffe, M. 2008, Electic cars ‘silent killers’ accessed on 29/08/08 at http://www.news.com.au/couriermail/story/0,23739,24224490-5010760,00.html Woodcock, A. and Scrivener, S.A.R.1997, User requirements and networked applications. In S.A. Robertson (ed) Contemporary Ergonomics, 516–521, Taylor and Francis: London.
RESEARCH AND POLICY DEVELOPMENT INTO SCOOTER ACCESS ONTO LONDON BUSES S. Thomas CCD Design & Ergonomics Ltd Access to the London Transport network is growing more important with the increasing use of motorised scooters by people with a range of mobility problems. TFL are concerned that the current policy on mobility carriage transportation on board buses is unclear and does not give enough clarity on which types, specifically scooters, that may access their bus fleet. Previous research cited in Department for Transport’s: Carriage of Mobility Scooters on Public Transport (2007) recommended that only scooters of a comparable size to conventional wheelchairs be allowed access onto public transport. CCD Design & Ergonomics ran and led the research programme and produced a final report detailing the findings.
Introduction Being committed to developing an accessible transport system for all, Transport for London (TfL) have ensured that their bus fleet is now completely accessible to wheelchair users through ramp access and specified wheelchair spaces on every bus in service in the Capital. With the increasing use of motorised scooters by people with a range of mobility problems it is now being argued, by some, that scooter access to the transport network is important to allow those with mobility impairments to be able to travel freely, and that this should form part of any accessible transport service, alongside wheelchair access. TfL are concerned that their Current Conditions of Carriage for buses do not give bus drivers or the general public enough clarity about the types of scooters which might be able to access their bus fleet. TfL therefore wished to undertake research to inform the development of a clear and consistent policy towards the access of mobility scooters onto the bus fleet for all the relevant stakeholders, including London bus companies, their drivers, users of mobility scooters, wheelchairs and the wider public. TfL envisaged that such research would establish which scooter models were suitable for access to the bus fleet, identify the key stakeholders and ensure that the conclusions could be communicated clearly to them. Previous research by the Department forTransport: Carriage of Mobility Scooters on Public Transport (2007) recommended that only scooters of a comparable size
122
Research and policy development into scooter access onto London buses
123
to conventional wheelchairs be allowed access onto public transport. It also identified further issues with the carriage of scooters that required clarification and which TfL wished to investigate by commissioning research specific to London Bus Services. CCD is a human factors and design consultancy specialising in the usable design of public transport vehicles, systems and infrastructure. CCD supported their research by drawing upon the facilities and capabilities of partners Ricability and MIRA, in order to develop a project team offering countless benefits. CCD ran and led the research programme and produced a final report detailing the findings. Ricability supported the scooter access user-trials design by CCD, specifically providing and organising access to their established panel of mobility impaired trials participants and securing suitable models of scooters.
Figure 1. Typical Scooter Models: Wispa, Sun-runner, Go, Go. Ricability has established contacts within scooter distributors to facilitate the borrowing of test scooters. Ricability’s panel of mobility impaired people includes scooter users and these formed a test group for user trials. To facilitate easy access for the user group to testing facilities, Intertek laboratories in Milton Keynes, with whom Ricability traditionally collaborate on research, were employed. CCD are very experienced in the design and build of bespoke mock-up and/or test rigs. Our approach to assessing the accessibility of various bus designs by scooters was to design and build an adjustable test rig, which could be easily and quickly re-configured to represent different bus layouts. The evaluations included: • Objective assessment of scooter access (and stability issues) to buses via ramp and manoeuvrability in mock-up bus vestibule spaces • Manoeuvrability of different scooter types into, around and out of current DfT compliant wheelchair spaces within the different bus configurations • Usability differences between scooter types (wheel configuration, seat shape/type and position, accessorised etc) and possible safety implications within current DfT compliant wheelchair spaces in buses
124
S. Thomas
• Scooter user opinions/experiences of access to bus types (i.e. subjective opinion of acceptability, risk and comfort) The project was also supported by MIRA, who provided computer-based simulation facilities to investigate the dynamic stability of suitable scooter models to quantify the risk of their tipping and/or moving during sharp turns or sudden braking, both in forward and rear facing positions and in comparison to wheelchairs that are already accepted for.
The brief The issues TfL wished to investigate included: • Good practice, including policies and procedures or codes of practice from other bus companies, and good practice in terms of communicating and disseminating revised conditions of carriage to the general public. • The application of the DfT wheelchair size recommendations to the London bus fleets and the ability of the current range of scooters to comply. • The manoeuvrability of different scooter models with respect to the designs of the different vehicles in the London bus fleets and the different locations of wheelchair spaces. • The health and safety risks associated with each model including the stability of different models (3 and 4 wheel models) with reference to the designs of the different vehicles and the locations of wheelchair spaces, and the risks associated with unsecured mobility vehicles when buses are in transit. • The issues faced by wheelchair and scooter users when travelling by bus and how new policy may address those issues, i.e. when is it appropriate to ask travellers to move from a scooter to a seat during transit? Does bus design make access difficult? • The views of London bus companies and their experiences so far in meeting/not meeting the needs of scooter users. • The implications of the findings to future design of compliant scooters. The research project produced the following outputs: • Understanding of the size and model restrictions in terms of the access of scooters onto the London buses. • Clear policy direction in terms of security and risk relating to different models of scooter when on board. • Guidance for manufacturers and retailers concerning the suitability of different scooters (design/size) for the bus service. • Understanding, via consultation, of the needs and views of users of scooters, buses and drivers/bus companies, which will inform the development of a clear and useable policy on the access of scooters. • Examples of good practice in communicating policy guidelines to the public and bus companies/drivers, including use of endorsements/kite-marks, etc.
Research and policy development into scooter access onto London buses
125
Methodology Research & survey work CCD undertook a literature and internet based search for examples of good practice, policies, procedures and codes of practice from other bus companies, both in the UK and abroad. CCD also used local contacts in USA, Canada, Australia and Europe to ascertain and review the policies and codes of practice being implemented by bus companies and in particular what means they have used in disseminating their conditions of carriage. Additionally, CCD contacted usability/mobility groups, discussion forums and websites in order to get user feedback on their understanding of terms of carriage, an understanding of where the gaps in their knowledge are, issues they have with both the terms and how they are made available and views on which means of dissemination they find to be most effective. CCD also surveyed the London bus fleet, building on data and measurements developed for previous studies into push chair access, to determine the variation in the range of space and facility provision, particularly in terms of door access, vestibule size, wheelchair space (i.e. size, layout, position in bus and provision of rest) as well as other potential impediments to manoeuvre such as seating, stanchions, poles etc. The range of vehicles currently operated includes double deck, single deck and articulated buses all with low floor capability and ramped access. While there are a fairly large number of bus models in operation, these tend to be based on a limited number of chassis types with bodies from a number of coach builders.
Stakeholder consultation TfL currently contract some 18 bus companies to provide services in and around London. To get the broadest coverage and exchange of views CCD held a number of interactive workshops at their London offices with representatives from each company. The workshops were aimed at understanding the needs and views of the bus companies, their experiences so far in meeting or not meeting the needs of scooter users, their views on operational issues (such as access protocols, delay to service, issues with the possible use of scooter restraints, checking/monitoring proper procedure by the scooter/wheelchair user etc) and safety risks and their methods and associated experiences of disseminating codes of practice (both to their staff and to the public). Each workshop involved invited company stakeholders responsible for policy and operation. Some 40 bus drivers, from various bus operating companies, were also interviewed on site during their shift breaks. CCD ran a series of user workshops across London to undertake a structured review of the needs and views of scooter and wheelchair users in relation to bus access, in particular to understand their requirements for access, their experiences with accessing buses and with how bus companies operate their policies, their views
126
S. Thomas
on the availability, clarity and dissemination of terms of carriage and their views on issues such as the provision and use of wheelchair/scooter restraints and transfer to a seat during transit. CCD directly contacted a number of relevant user groups (such as Transport for All and Shopmobility) with the aim of gaining wider access to user views with a questionnaire survey. CCD also completed user interviews at the Mobility Road Show (12th to 14th June 2008). The show is the largest annual mobility and lifestyle event in the world and attracts thousands of people with mobility problems from all ages who take the chance to assess products and services available to help them. The show offered unrivalled access to the widest range of interested and motivated potential mobility impaired users of bus services.
Access and manoeuvrability testing CCD assessed the size and model restrictions on scooter access and the ability of various scooter models to comply with DfT recommendations by a series of user trials using various models of scooter on a mock-up bus test rig. CCD specified an adjustable bus rig capable of being re-configured to represent the different examples from the London fleet including a ramp, doorway, vestibule area and wheelchair bay as well as any poles and stanchions etc. that might affect scooter access. The aims of the trial were as follows: • to identify which models of scooter can be manoeuvred by real users onto a bus and into a wheelchair space • what aspects of bus and/or scooter design most impact on or restrict manoeuvre/ access (especially scooter turning circle and wheel base and relative positions and design of bus structures) • what elements of access might pose risks to scooter or other bus users (i.e. internal manoeuvre, sudden speed increase at the top of the ramp, stability on or leaving the ramp etc) • whether the manoeuvrability of different models of scooter within the bus or their mode of entry limits them to facing forward or rearward when the bus is in transit CCD believed that this last issue is of some significance, since previous research and policy in both Europe and Canada suggests that scooters and wheelchairs provide safer seated position in transit when facing the rear of the bus. Indeed the majority of wheelchair spaces provided in buses are designed for rearward facing occupation. CCD believed that it was important to understand whether some models of scooter that might otherwise fit in terms of size, may not be capable of being manoeuvred into a wheelchair space to face rearward.
Health & safety assessment As part of the stakeholder consultation workshops (as described above) CCD ran hazard identification and hazard and operability sessions (HAZID and HAZOP)
Research and policy development into scooter access onto London buses
127
Figure 2. Test Rig.
to identify and quantify perceived health & safety risks associated with the carriage and access of scooters on buses and, where possible, identify options for risk mitigation and design improvement. This provided a detailed understanding of the level of risk and concerns of safety from the perspectives of both bus companies, their drivers and of the service users. As part of the user trials of scooter, access and manoeuvrability potential health and safety risks associated with each model (3 and 4 wheel) that meets TfL’s wheelchair space requirements were identified, especially in relation to stability
Figure 3.
Mock up.
128
S. Thomas
during ramp access and subsequent internal manoeuvre and in relation to potential interaction with other bus users. Of particular interest to TfL (and as highlighted in previous research) is the potential risk of tipping, particularly for 3 wheel scooters, when the bus is in transit. CCD investigated dynamic stability and potential issues with sudden swerves or braking by computer-based simulation. CCD employed MIRA’s Simulation and Safety Engineering department to make a series of simulation tests with their MADYMO 3D system of those scooter models which meet TfL requirements, each with occupants of various masses (e.g. 5th, 50th and 95th percentile). The occupied scooter models were subjected to a series of defined accelerations from cornering, swerving, braking and accelerating from which the potential tipping or other types of response from the scooter and occupant were determined. Importantly MIRA’s assessments were compared with responses between forward facing and rearward facing positions, in order to determine the validity of current practice and research which suggests that rearward facing positions for scooters are safer. Following the simulation assessment, CCD made a design based assessment of the various buses in operation with a view to commenting on or developing mitigation to those risks identified, such as tipping prevention devices and/or alterative types of restraint that might be required. The aim was be to determine what, if any, measures might feasibly be employed (in terms of adaptation or process) to enable the carriage of models of scooter that present too high a risk of tipping, but might otherwise meet TFL’s requirements.
Results Research & survey work Mobility Road show The most interesting findings from the road show were from non-London transport users and the reasons why they chose not to use their scooters on public transport and buses. The following were a few of the reasons: inaccessibility, they owed a disability vehicle, crowding issues, confusion on policy, previous bad experience.
Focus groups and forums See ‘Stakeholder Consultation’ below.
Survey of London bus fleet In total, eight models of bus were identified by TFL as those with the most limited interior space in terms of the wheelchair allowance, infrastructure such as poles, etc. These models were then photographed, measured, assessed and ranked in terms of space limitation, the worst five going through to pilot stage.
Research and policy development into scooter access onto London buses
129
Stakeholder consultation Managers and Drivers. Both managers and drivers were of the opinion that the current situation is that scooters are allowed onto buses at the discretion of the driver in the absence of any firm guidelines; and that there also seems to be a lack of guidance on what happens if a wheelchair/scooter needs to board and the bay is occupied by a wheelchair, child buggies, which may or may not be collapsible, etc. In any event, the result could be confrontational with the driver. There are a multitude of motorised scooters of different sizes and configurations (3 wheel, 4 wheel etc) and the suitability of allowing the whole range is unlikely. There appears to be three main options, all of which will not be welcomed by some Stakeholders: • Disallow all scooters • Allow all scooters • Allow “compliant” scooters by some form of “badge” scheme. Drivers felt that to allow scooters onto a bus, they would have to present minimal additional disruption when compared to a wheelchair. In general a wheelchair can enter the bus by the ramp powered by the occupant or an assistant. It can then turn within its own length, reverse into the bay, rest against the backboard and apply a brake to stop movement during motion of the bus. Exit is a direct drive out. On entrance, if the bus is unable to get close to the kerb then it is likely that the wheelchair can still get off the kerb and access and exit the ramp from road level. Drivers would therefore suggest that to allow access to scooters, then certain criteria need to be met to ensure that they are comparable to a wheelchair for both safety and boarding times. Users. It was evident from the focus groups and forums that the majority of users interviewed had, at least, attempted to use their mobility scooter on public transport. Those who had not would certainly like to be able to as they felt it would enable them to be more independent. A few of these users had attempted but were refused by drivers for reasons such as too many people currently on board, not being allowed to carry them, the model in question was too big and so on. Drivers were often being quite rude and users stated that they are made to feel that they are a burden to the public. Of those who had and do use their scooters on public transport, the majority felt that it gave them more freedom. The majority of users also claimed that they had travelled in the forward facing position and not rearward as specified in the manual wheelchair regulations. Users felt that by travelling forwards it enabled them to establish their position and right to travel on the bus, they didn’t have people staring at them, they were able to see where they were going and (in the absence of an I-bus) could see when to get off. Users felt strongly that there was no need to travel in the rearward facing position, since the seat pad was not designed for a scooter. Users felt that should the policy be changed, it would not be possible for all models of scooter to travel on the bus and those models that could, would need to
130
S. Thomas
be identified somehow to users and drivers. The issue of prioritisation would also need to be addressed in terms of buggy, wheelchair users and the elderly. There would also need to be re-instruction to bus drivers as many users felt that they were ignorant to their needs and to all bus users.
Conclusions • It is clear that scooter users would generally be keen to travel on London buses and would benefit from doing so. Usage is currently limited because users do not believe they are allowed on buses, because they do not know that buses might be accessible to their scooters or because drivers refuse to take them, for various reasons. • The user trials have demonstrated that a number of scooter types can access to some buses, although clearly not all models are compatible with all buses. • Accessibility for scooters is affected by a range of factors. Just because a scooter’s physical dimensions fit within the standard wheelchair space does not mean it can be carried. Their manoeuvrability, the shape and position of their wheels and handle bars, their floor clearance and so on interact with the access ramp, available space and various structures in the bus in order to limit access. Additionally, time taken to access/egress is likely to prove an important additional limiting factor. Some scooters require a lot of manoeuvring to access the wheelchair space (typically multi-point turns) which takes a considerable time. Clearly, in trying to operate a timetable, TfL may be forced to exclude some scooter models simply because they take so long to enter or exit buses that they would seriously affect the ability to run a timely service. • The policy for the carriage of scooters will have to clearly identify those scooters and buses that are compatible. Due to the range of factors influencing accessibility, it is likely that TfL will need to test future models and keep a register of those that are suitable for carriage, updating it as new models of scooter and bus evolve. • The most significant limitation to access in terms of bus design is the position stanchions/handrails around the wheelchair space. Opportunities for removing, re-designing or re-positioning these should be considered in order to improve accessibility, especially in reducing the time it takes to manoeuvre in and out of the wheelchair space. • Rearward facing travel (for the sake of safety) is unlikely to be feasible and if made a policy requirement would seriously limit the range of scooters that could be carried. Few scooters are sufficiently manoeuvrable to turn around inside the bus in order to reverse into the space, and given both the limited width of the ramp, limited capability of scooters and the nature of users (many suffer from restricted joint mobility and cannot easily turn around) it is not feasible or safe to expect users to reverse on to the bus. Additionally, it is likely that many users would be capable of transferring from their scooter to a bus seat (the bus must remain stationary). It is also thought unreasonable that scooter access be restricted on this basis given that wheelchair users often (if not typically) travel
Research and policy development into scooter access onto London buses
•
• •
•
131
facing forward, despite policy to the contrary. It would seem unfair not to allow scooter users to face the same risks as wheelchair users choose to. In addition to defining the range of scooters that can be carried on certain buses, the carriage policy will need to clearly and unambiguously explain the conditions of carriage, safety issues and limitations. For it to be successful, the policy needs the widest possible dissemination, including at least relevant websites, scooter manufacturers, mobility advisory bodies and assessment centres, mobility forums and groups and so on. It will also be important to reach the widest possible audience, especially the non-scooter using public, who also need to be made aware of the policy if it is going to be effective and accepted. Drivers need to be properly and carefully trained on the policy itself and how to improve access through judicious choice of ramp deployment, announcements to passengers to make way for scooters if the bus is crowded, etc. Drivers need an easy way of identifying compatible scooters. The simplest method will be badge or flag on the scooter to allow the driver to see clearly that a given model is compatible with their bus. For this to work it is likely that TfL will have to employ some form of registration scheme whereby users apply for the identifying mark declaring the model of scooter they use. There are likely to be some operational problems that will need resolving if the policy is to operate smoothly. Drivers will require some means of indicating to waiting scooter users that the wheelchair space is already legitimately occupied (e.g. an external indicator light on the bus or external PA facility).
References Department of Transport, 2006, Carriage of Mobility Scooters on Public Transport Feasibility Study. http://www.dft.gov.uk/transportforyou/access/tipws/cmspt/ (at 21 October 2008). Department for Transport, 2000, Public Service Vehicle Accessibility Regulations (PSVAR) 2000 – Guidance.
HAZARDS
STUDY OF VERBAL COMMUNICATION BETWEEN NUCLEAR PLANT CONTROL ROOM OPERATORS DURING ABNORMAL SITUATIONS A.N. Anokhin & N.V. Pleshakova Obninsk State Technical University for Nuclear Power Engineering, Obninsk, Russia Communication plays an important role in the joint mental activity of Main Control Room (MCR) team members at a Nuclear Power Plant (NPP). Today a study of communication is considered valuable not only as a means for investigating information exchange, but also as a tool to improve personnel reliability and for reducing human errors. In the present work, the authors performed a detailed analysis of the communication which takes place when operators are dealing with an emergency situation at NPP. Based on interviewing the operators, the analysis of emergency procedures, and by conducting of a series of experiments, the following outcomes were obtained: six types of communication were identified, the amount and relative proportions of these various types communication were estimated, the factors influencing the communication were revealed and assessed, and recommendations of ways as to improve communication within the MCR were formulated.
Introduction Communication is defined as the exchange of information while preparing for, or performing work, during NPP control (Barnes et al., 2001). Purposes of communication include maintaining a common understanding between team members and thereby increasing operator reliability. Numerous researches have considered communication and its influence on the effectiveness of teamwork. Studies by Jentsch et al, (1995) and Johannesen et al, (1994) found that teams, who were more effective in their performance, used more standard communication and vocalized more situation awareness observation relative to changing important process parameters and alarms. Communication can result in a unified and accurate team situation model, that, in turn, aids collaborative action planning and is then needed to adjust strategies or to develop new ones to deal with the situation (Ma et al., 2006). Schraagen and Rasker (2001) studied the role of communication in the development of the shared mental models of the process situation. Their results showed that communication is not so beneficial in routine situations, when team members know exactly what is going on. In novel situations, however, communication is necessary for operators to respond to environmental cues, to explain to each other why previous strategies will not work in the current, novel situation and then it 135
136
A.N. Anokhin & N.V. Pleshakova
permits co-operative determination of new strategies and the prediction of future states. Increasing the level of automation can influence and improve communication between team members (Johannesen et al., 1994). However, new control room designs also can impose challenges on team working. Norros and Savioja (2004), noted, that transition to ‘soft control’ can diminish the horizon of observation and hinder information exchange. One of the methods for measuring team communication is to categorize communication types and then quantify communication in accordance with the categorization. Schraagen and Rasker (2001), for instance, considered and estimated different communication categories such as information exchange, determining strategies and performance monitoring. Barnes et al (2001) studied the communication errors and showed that a verification feedback serves an important error-checking function in the communication process. Through feedback the similarity of the meanings given to the message by the sender and understood by the receiver can be verified. An example of feedback in verbal communication is when the receiver ‘repeats back’ the message and the sender either agrees with the receiver’s repeat back or corrects it. Today, communication more and more plays an additional role in that it is used as a means for increasing the reliability of human operations and particularly, to avoid operator errors. After the series of accidents at NPPs during the 1970s and 80s the emergency management procedures tend to be more strictly formalized and structured. In order to implement the next step in a procedural sequence, the unit shift supervisor has to read the required step’s wording out loud. Then, the appropriate operator (reactor operator or turbine operator) executes this step and reports back about any obtained results. Such a protocol, contributes to increasing the reliability of operator actions. However, this kind of communication is superimposed on the regular exchange of information between MCR operators, field operators, plant specialists and power grid dispatchers. This requirement thus increases the frequency and intensity of communication traffic which can overload the information transmission between operators. As a rule, the emergency procedures are used under stress conditions and under time pressure. In such situations, the communication in itself may be subject to errors and the operators are forced to find ways for optimizating the message exchange. This paper is devoted to the analysis of the communication which takes place between NPP MCR operators when managing abnormal events. The paper considers the categorization and the structural models of various types of communication acts, the estimation of relative proportions of the various communication types, the analysis of factors preventing communication, and a description of methods used by operators in order to facilitate communication.
Methodology The study comprised three stages, namely: interviews with operators, analysis of emergency procedures, and an experimental study. The reference NPP was a Pressurized Water Reactor (PWR-1000) unit generating 1000 megawatts of electrical
Study of verbal communication between nuclear plant control room operators
137
power. The experienced MCR shift personnel were interviewed at the first stage of the analysis. Most of operators emphasized that they felt a lack of time when required to read the wording of a procedure step by step, instead of immediately implementing the required actions in a rapidly evolving emergency situation. Then the relevant emergency procedures were studied. Two broad kinds of procedures are used at NPPs, namely ‘symptom-oriented’procedures and ‘event-oriented’ procedures. Procedures of both types are represented in a two-column tabular format. The third stage, the experimental study, consisted of observation of MCR teams while managing an emergency scenario presented using a full scope NPP training simulator. The scenario reproduced the superimposition of two faults – a leakage of water from the primary loop to the secondary loop inside the steam generator and a leakage of water from a primary loop into the containment. Four MCR teams took part in the experimental study. Each team consisted of operators who have had some years of work experience in the MCR. Each MCR team consists of four persons, namely – reactor operator, turbine operator, shift supervisor of the reactor plant and the unit shift supervisor. Two teams used the symptom-oriented procedure in order to overcome the situation, and two teams used the event-oriented procedure. The actions performed by operators were recorded by video camera, and all relevant process parameters, manipulation of controls and technological events (alarm signals, switching of equipment) were registered by the simulator computer. A hierarchical task analysis and timeline analysis were carried out. All verbal messages from the MCR operators (160–180 messages in each simulator run) were categorized and their expected duration was estimated. The following work has been undertaken in the course of the study: • the main types of communication were revealed, categorized and described; • relative proportions of the various communication types were recorded. • the performance shaping factors and the main problems were considered and analyzed.
Types of communication acts MCR operators communicate both with each other and with outside personnel. Among themselves, operators communicate verbally, sometimes accompanying their messages with gestures. Outside participants are field operators, shift supervisors of various departments, the NPP shift supervisor, NPP senior managers, and the dispatchers of various systems – power grid, fire brigade, etc. MCR operators communicate with outside personnel by means of telephone or over the public address system. An elementary component of communication is an act of information transfer from one sender to one or more receivers. In the more complex case, communication takes place in the context of some action. Such a communication includes two or more information transfer acts (for example, question-answer). Figure 1 presents a structural flowchart for the act of information transfer. In the simplest case, such an
138
A.N. Anokhin & N.V. Pleshakova
Figure 1. A structural flowchart illustrating the act of information transfer from sender to receiver.
act includes just one activity element, namely: the sender speaks the information. In the more complex case, the receiver provides feedback and either confirms adequate perception of the information, or asks for repetition or to clarification of the information. The sender repeats the message or provides some additional data. After the exchange, the act may be completed or continued until the transferred information is satisfactorily perceived by the receiver. Information transfer acts can be categorized into three generalized types depending on the purpose and content of the information to be transferred, namely: 1) Query – The sender asks the receiver for certain information; 2) Report – The sender informs the receiver about certain events; 3) Command – The sender instructs the receiver to perform certain actions. The communication act is a combination of the above mentioned three types of information transfer act. Figure 2 illustrates the six various types of communication, between NPP MCR operators, that were revealed during the study. (a) An Information report consists of one act of information transfer to one or more receivers. Usually an information report is initiated by a reactor or turbine operator (or field operators through communication facilities). As a rule, such a report contains information about incoming alarms, the deviation of process parameters and any change in technological equipment state. (b) A Query-report includes two information transfer acts. In most cases, the MCR shift supervisor asks operators for necessary information or the MCR operators issue a query to field operators. A query can by expressed either as an interrogative sentence (for example, ‘what is the current concentration of boric acid?’) or as an instruction to perform an inspection action (for example, ‘Check the current concentration of boric acid‘). A Query implies an obligatory reply.
Study of verbal communication between nuclear plant control room operators
139
Figure 2. Flowchart diagram for the six types of communication: (a) information report, (b) query-report, (c) discussion, (d) report about action performed without command, (e) command-action, (f) request to perform action.
(c) A Discussion takes place when the receiver and sender exchange information and try to find a root cause of situation. During such mutual discussion, members of the MCR team usually clarify the current situation or make a decision about any necessary joint action. (d) A Report about action performed without command takes place after the reactor or turbine operator has independently performed a certain technological operation. The purpose of this type of communication is to maintain the situational awareness of other team members. (e) A Command-action. This type of communication commences with an instruction to perform a certain technological operation. Usually, the unit supervisor is the initiator and sender of the command, whereas the reactor or turbine operator is the receiver. In some cases, the command can be initiated by an MCR operators and will be addressed to the field operator. After the command has been issued, the receiver performs the prescribed operation (usually this is a motor action – such as switching on a pump, controlling a valve, etc.) and then reports back as to the outcome. In some cases, the report may be omitted. (f) A Request to perform an action is initiated by operators when they need an approval of this action from the plant or unit supervisor. After the unit supervisor has received the query, he appraises the situation and then either approves or rejects the action. Upon receiving approval, the operator completes the action, after which he can then report on the results. In total, 529 acts of communication were identified during four run-throughs of the abnormal scenario and their total duration was 2 hours and 17 minutes. Communications took up about half of this total time. The average duration of one communication act was 7.7 sec. Table 1 shows the quantitative characteristics of the different
140
A.N. Anokhin & N.V. Pleshakova
Table 1.
Proportion of the various types of communication.
Sort of communication
Relative proportion of the communication type, %
Relative time spent on this type of communication, %
Average duration, sec
Information report Query-report Discussion Report about action performed without command Command-action Request to perform action
38 23.5 5 3.5
16 28.5 12 4.5
3.3 9.4 18.4 10.1
24 6
34 5
10.9 6.8
communication types. The table shows that the majority of the communication time was occupied by information reports, query-reports and command-actions.
Factors which influenced communication During the running of the scenarios and the analysis of the video records, it was revealed that a considerable proportion of information transfer acts included requests to clarify or repeat information (see Figure 1 for the structure of each act). A High level of noise within the MCR was identified as one of the main causes. During abnormal situations, the noise is produced by activating audible alarm signals and the extremely intensive conversations between the operators (particularly between MCR operators and external personnel). Each alarm is accompanied by an audible signal such as a loud ring or a siren. Though the operators try to avoid talking when alarms are sounding, the signals often occur after the communication act has started. In the case of a rapidly developing situation, the frequency of alarms is very high and operators have insufficient time for the exchange of messages in the extremely short breaks between consecutive alarm signals. The frequency of communication and the frequency of audible alarm signals during the four emergency scenario runs are shown in Figure 3. In most situations there is a low correlation between the two curves (the Pearson correlation coefficient is 0.3–0.7). This is to be expected because the operators must report about the deviation of process parameters and any change in equipment operational mode immediately after activation of the appropriate alarm signals. Moreover, when the situation is developing rapidly, the operators make a joint effort to find an explanation for the occurring events and to develop a suitable strategy of actions. In order to be heard, the operator is forced to raise his voice and to shout above both the sound of alarm signals and over any talking between other team members. Another cause of excessive noise in the MCR can be the presence of additional personnel such as the safety engineer, shift supervisors from various departments, the nuclear physicist, etc., who may all be required in certain situations. Additional personnel can arrive at the MCR both during the abnormal situation or in the
Study of verbal communication between nuclear plant control room operators
141
Figure 3. Frequency of communication and audible alarm signals when dealing with the emergency scenarios (——– – Frequency of communication acts, - - - - - – Frequency of audible alarm signals).
case of scheduled changes to the unit operational mode (for example, raising the reactor to criticality). The additional personnel may assist the MCR team members to solve problems and may partially relieve the reactor and turbine operators of their high workload. However, these external specialists can increase and overload communications and inevitably raise the level of noise in the MCR. The second, essential factor preventing fast and accurate communication is the occurrence of several communication acts together. Superimposition is the appearance of two or more communication acts concurrently. Most often superimposition takes place when several, different senders try to transfer information to one receiver. For instance: • when the state of the unit equipment is changing very rapidly and both operators (reactor and turbine) report main events to the unit shift supervisor at the same time; • when the shift supervisor issues a command to an operator to perform a certain action while this operator is already busy talking with a field operator. In the case of the superimposition of communication acts, the receivers often asked the sender to repeat the message. The same situation was observed when an audible alarm was superimposed on a communication.
142
A.N. Anokhin & N.V. Pleshakova
A third essential factor influencing communication is the information content of the transferred message, query and command. In accordance with the existing rules for the implementation of emergency procedures, the unit shift supervisor has to read out loud the whole of each sequential step of a procedure, following which, the appropriate operator (reactor or turbine) must carry it out.. It had been revealed as a result of a previous ergonomics assessment of procedures, that most of the procedure steps, as formulated by ‘written language’, were much more ponderous and verbose when compared with colloquial language. The wording of a procedural step typically contains complex, technical equipment identifiers, such as 4RT11,12,13,14S11, which can be awkward to perceive verbally although they can be read without difficulty. When interviewed, the operators said that, under the conditions of time pressure and in a rapidly expanding emergency situation, reading all instructional steps aloud and verbatim, actually retards the operator’s response to the situation. Analysis of video records revealed that when the personnel were in a hurry they cut out the wording of some steps and omitted unnecessary and redundant words. When operators mentioned some equipment items and actions, they often used professional slang expressions which have been established as a result of longterm, joint working within a team. Such a use of slang enables the reader to save time and to reduce the duration of reading of procedural steps, on average, of three times. This assessment was obtained by measuring (using the video recordings) the duration of each communication act, including reading of the instruction and then comparing the measured duration with the expected duration. In order to estimate the expected duration, each instruction was read aloud verbatim five times. Each time the reading duration was measured to an accuracy of 0.1 sec and the average time was calculated. In a number of cases, the unit supervisor himself performed the operation of checking of a process parameter, rather than reading out the relevant step of procedure and then seeking the value of this parameter from the operator. This type of task substitution reduces the duration of step performance by, on average, a factor of five to six. At the same time, the reduction of wording and the usage of slang can lead to ambiguous interpretation of the content by the receiver. Use of slang to refer to technological equipment may force operators to perform a mental conversion of information because all control panels and documentation use only “official” labels and identification. Obviously, this increases the cognitive workload on operators and creates the preconditions for mistakes. However, it should be noted that, during the conducted experiments, no such errors were observed.
Additional findings and recommendations Along with the analysis of communication, the way in which operators restored the power unit from transient, abnormal, state to normal (stabilized) mode was also studied. An evident relationship was revealed between the time spent by MCR personnel for communication with each other and the overall time spent to normalize
Study of verbal communication between nuclear plant control room operators
143
the situation at the unit. The more time operators spend in communication, and the more carefully and accurately they transmitted information between each other, then the more quickly, smoothly and effectively they achieved the normal state of the unit. This conclusion is in agreement with similar results reported by Jentsch et al (1995). This dependency is predictable, since the communication enables the team to maintain awareness about process events and about the actions performed by all team members. Nevertheless, if the amount of communication exceeds a certain critical limit then operators may lose the time required to perform the necessary actions. It was also noted that senders often call the receiver by name (or name and patronymic, as is customary in Russian) at the beginning of an information transfer act. This immediately stimulates the recipient and concentrates their attention, thus allows them to avoid superimposition of messages and reduces the need for clarification questions. Another way to improve the efficiency and reliability of communication is by read back, when the receiver confirms that he/she has received and interpreted the information in the correct way. Most often, the receiver simply repeats back the received information. Undoubtedly joint planning and decision-making regarding the required followup actions can be very useful. Joint cognitive work can often lead to more effective teamwork. However, it is not always so. For example, in one conducted experiment, an operator incorrectly identified the initial event and then persuaded the unit supervisor to follow an inappropriate procedure. The following recommendations, which are aimed at improving communication have been formulated as a result of the analysis of findings. 1. There is a need to adjust not only the volume, but the tone of audible emergency and warning alarm signals in the MCR. It is important to ‘divorce’ the tone of the audible signals from the frequency and tone of the human voice. 2. The wording of procedural steps were more easily perceived when rewritten particularly when complex linguistic constructions were replaced. 3. It would be useful to unify and standardize the use of slang terminology in order to develop an unambiguous description of certain actions and technological equipment. 4. Provide the unit supervisor with contextual technological information in which the content depends on the current conditions and procedural step. This would particularly reduce the number of ‘query-report’ acts and would lead to a general reduction in communication load. 5. In order to avoid the loss of important information in case of superimposition of communication acts, it is necessary to concentrate the attention of the receiver to high priority messages (for example, by using the keyword ‘important’).
References Barnes, V., Haagensen, B. and O’Hara, J. 2001, The human performance evaluation process: a resource for reviewing the identification and resolution of human performance Problems (NUREG/CR-6751), (U.S. NRC, Washington, DC)
144
A.N. Anokhin & N.V. Pleshakova
Jentsch, F., Sellin-Wolters, S., Bowers, C. A. and Salas, E. 1995, Crew coordination behaviors as predictors of problem detection and decision making times. In Proceedings of the HFES 39th Annual Meeting, (HFES, Santa Monica, CA), 1350–1353 Johannesen, L.J., Cook, R.I. and Woods, D.D. 1994, Cooperative communications in dynamic fault management. In Proceedings of HFES 38th Annual Meeting, (HFES, Santa Monica, CA), 225–229 Ma R., Kaber, D.B., Jones, J.M. and Starkey, R.L. 2006, Team situation awareness in nuclear power plant process control. In Proceedings of 5th ANS International Topical Meeting NPIC&HMIT 2006, (ANS-Omnipress, Washington DC), 459–462 Norros, L. and Savioja, P. 2004, Usability evaluation of complex systems. A literature review (STUK-YTO-TR 204), (STUK, Helsinki) Schraagen, J.M. and Rasker, P.C. 2001, Communication in command and control teams. In Proceedings of 6th Command and Control Research and Technology Symposia, (CCRP, U.S. Department of Defense, Washington, DC)
. . . IT’S THE WAY THAT YOU DO IT: ENSURING RULE COMPLIANCE J. Berman, P. Ackroyd & P. Leach Greenstreet Berman Ltd, Reading RG1 4QS, UK Ensuring compliance with procedures and process is of great importance not only for high-hazard organisations, but also in other sectors such as Finance. Whilst much is known about the causes of noncompliance, such as inadvertent actions, misunderstandings or deliberate violations, non-compliance remains a challenge. The challenge for industry is to understand the potential causes of non-compliance and to identify appropriate measures to defend against them. The solutions frequently lie in areas of conventional ergonomics – task, interface and procedure design, and operator competence. And yet organisations frequently find difficulty in seeing a clear way forward. The importance of an ergonomics approach is highlighted, but as part of a coherent strategy for managing rule compliance, including supervision and leadership skills. It also emphasises the contribution this makes to creating a resilient organization.
Introduction What do the ‘rogue traders’ at Société Générale and Barings Bank, and the explosions at the BP Texas City refinery and the Buncefield oil storage depot in Hertfordshire have in common? They were all either caused, or aggravated, by failure to follow procedures. Ensuring compliance with procedures and process is of great importance for all organisations – whether traditional ‘high-hazard’ or economic high-consequence. Much guidance has been written concerning noncompliance, ranging from explicit guidance (e.g. HSE, 1995) through to general guidance on ergonomics, procedure design, etc. and hence it could reasonably be assumed that the problem would now be properly controlled. The reality is that many organisations fail to ensure compliance with those processes and procedures that they believe are critical to their organisation’s commercial or safety performance. This begs the question – why is it apparently obvious (albeit with hindsight) what should be done, but apparently so difficult to do it? This paper discusses how the issue can be manifest, and the challenge of identifying and implementing effective solutions. Whilst many solutions lie within the domain of ergonomics, selection of the appropriate approach requires an understanding of the goals that underpin the observed behaviour. The paper considers the nature of non-compliance, the importance of the concept of goals, and the idea of a contract between the individual and organisation. This leads to consideration of the manner in which non-compliant behaviours are manifest (rather than focussing 145
146
J. Berman, P. Ackroyd & P. Leach
solely on the actual breach of process). The paper then considers what some of the solutions might look like, and approaches to proactive improvement. This is then discussed in the context of resilience, and the manner in which organisations can continue to move forward.
What is the issue? It’s important to understand the nature and extent of non-compliance – what we mean by the term ‘non-compliance’ and hence what we are trying to address. It’s easy to consider a set of behaviours as representing a common set of deficiencies, and hence being amenable to a single set of solutions, when in practice they encompass a wide variety of shortfalls and challenges. Compliance is concerned with the willingness and ability to follow agreed processes and procedures. It is important that the agreed processes, whether or not they are formally described in procedures, are acknowledged both by the organisation and by those charged with undertaking the tasks and activities, to be the required manner by which the desired outcomes are achieved. It would therefore appear that ‘non-compliance’ should be self-evident – the failure to undertake tasks in accordance with the agreed processes. Unfortunately, the reality is often less clear. What is the agreed process? Is the observed action a failure to follow the process or a legitimate deviation? What is the person attempting to achieve? What is the difference between non-compliance and a simple slip or mistake? What is the role of poor operator knowledge? The cause of non-compliance cannot be directly observed – only inferred from the context. Non-compliance is concerned with intended actions which deviate from an agreed process or procedure for a given task (i.e. it excludes simple errors). Three attributes are important: • Whether the deviation is deliberate – this distinguishes between active noncompliance (‘I choose not to follow a particular instruction’) and passive non-compliance (‘I was not aware that what I was doing was a deviation from the accepted procedure’). Passive non-compliance is particularly amenable to ‘ergonomics’ solutions. • The ‘correct’process is agreed – there is a contract between the individual and the organisation in respect of undertaking a task in a particular manner. The notion of contract implies obligations on both sides. Compliance must be feasible (all of the elements of ergonomics that contribute towards successful task performance), and the organisation must ensure that these elements are maintained in an effective manner. • Shared individual and organisational goals – compliance is concerned with voluntary behaviour by individuals, and hence it is important to understand the extent to which the individual’s goals align with the organisation’s. There will always be a disparity, but the greater the disparity, the more likely it is that non-compliance will occur. Understanding the disparity helps to identify where ergonomics can support solutions, and tends to be a key element in determining effective compliance management.
. . . It’s the way that you do it: Ensuring rule compliance
147
Given these attributes, there is a set of interrelated factors that can lead to non-compliance: • Culture: there may be attitudes within the organisation that affect the perceived importance of the implied compliance contract, or the extent to which it applies. • Goals: there needs to be sufficient consistency between the organisation’s goals and those of the individual. • Interfaces: the provision of information to the operator, and how the operator can execute the tasks affects their willingness to undertake the tasks in a given manner. • Procedures: the appropriate approach to undertaking the task can be agreed only if it is suitably described and communicated. Furthermore, the agreement requires more than merely being informed of the required approach – it must also be agreed as being feasible, viable, effective and required. • Training: the organisation must ensure that the people charged with undertaking the task are suitably equipped to do so in terms of knowledge, competence, and understanding of the context in which it is required. • Job design: the demands of the job must be reasonable and realistic. If an organisation is to address non-compliance effectively, it needs to consider how all of these issues come together, and to address them holistically and with the cooperation of those most directly affected – the ones who are apparently failing to comply. McDonald (2006) cites an example of a study of aircraft maintenance where one third of the technicians reported that the procedures do not present the best or safest approach and hence are not followed. What is notable about this is not only the proportion of staff who fail to comply with procedures in a highly regulated environment, but also the lack of surprise expressed by other in the industry when advised of this – it implies not only that non-compliance may be endemic, but that it is, in effect, condoned. This is not to suggest that the industry is unsafe – its record is very good – but that there appear to be many factors contributing to the choice of behaviours exhibited by staff who are apparently expected to exhibit high compliance. It should be noted, of course, that there may be very good reasons for the procedures as they are which are merely not apparent to the maintenance technicians. A concern is the apparent acceptance of a ‘pick-and-mix’approach as to which procedures require compliance. We have observed similar phenomena in such sectors as the nuclear industry where safety assessments have elicited such comments as “yes – that’s what the procedure says but actually we don’t do it that way”.
How is it manifest? At one level this may seem an odd question – it is surely obvious when staff fail to comply with a procedure. In practice, the obvious indicators are merely that there is a problem – but not what are the causes of the observed problem, and hence there is confusion as to effective solutions. An ergonomics approach (considering the entire work context and environment) would suggest that looking beyond the immediate issue is potentially beneficial, particularly in supporting a proactive approach to
148
J. Berman, P. Ackroyd & P. Leach
avoiding non-compliance. The question therefore is not “how are non-compliant acts manifest”, but rather “how are the organisational behaviours manifest which indicate a vulnerability to non-compliance”? • Culture: How does the organisation ‘reward’ different behaviours? What standards are demonstrated? What resources are available to revise work practices? • Goals: How does the organisation communicate its goals, and monitor (and support) the goals of its employees? • Interfaces: What evidence of interface shortfalls is sought within the organisation? How are interface concerns raised and addressed? What is the incidence of ‘special’ tools and job-aids? What is the evidence of ‘DIY’ modifications? • Procedures: What level of training is being implemented to compensate for potential procedural deficiencies? How is procedural guidance presented to the users? How are changes to procedures promulgated? • Training: How is training assessed? Have training syllabi kept pace with developments in the task and equipment? • Job Aids: Are there simple job aids that could be introduced? Is there reliance on ad-hoc job aids to compensate for poor task design? If the nature and cause of non-compliance is properly understood, then the set of potential solutions starts to emerge. Non-compliance is not inevitably associated with failure – many instances will occur without harm arising, and potentially with what appear to be positive outcomes such as reduced time or resources needed, such that the behaviour might be perceived as acceptable. It may be only after some time that a failure arises and the organisation focuses on those behaviours. It is therefore not sufficient to rely only on conventional indicators of performance. The organisation has an obligation to provide suitable monitoring arrangements – and this is part of the ‘contract’ between the organisation and staff. One set of indicators of non-compliance, ironically, might include evidence of apparent good performance. Why is a task now being completed in half the time? Why with significantly reduced cost? Of course this is not to say that any instance of good performance is indicative of an underlying violation. However, it reinforces the importance of investigating and learning from good performance. This also is part of the contract. If the good performance arises from a valid and appropriate alternative arrangement, then the lessons should be exploited. Alternatively, if it arises from an inappropriate approach – perhaps a failure to undertake an important safety check due to a perception that it is unnecessary – then addressing that deviation before it leads to an incident is clearly sensible. The individual instance may provide a leading indicator. It may also provide information about diverging goals.
What do the solutions look like? There are no generic solutions. There are many reasons for non-compliance, and hence each situation will demand different approaches. All too often the
. . . It’s the way that you do it: Ensuring rule compliance
149
Make compliance easier/remove problems Apply discipline/ sanctions/ rewards
Make rule better/correct
Involve staff in rules and implementation
Set clear standards Improved compliance Introduce a behavioural modification approach
Educate
Make rule and compliance important
Make non-compliance difficult/impossible Supervise/ monitor
Figure 1.
Potential solutions.
organisation’s reaction is to pursue training solutions, and perhaps procedural enhancements (assuming that they do not merely adopt a disciplinary approach). Whilst these might be of value, there are potentially many other solutions that might be relevant in certain circumstances. Figure 1 is derived from a toolkit developed for the rail industry (Berman et al., 2005), and indicates the range of potential solutions that could be considered. A key element of the toolkit is support for solution selection and implementation. It is rare that only one solution would be adopted, particularly when addressing compliance generically rather than merely seeking to prevent the recurrence of a specific instance. Furthermore, solutions that work in some instances may not be appropriate in others. For example, setting clear standards where the non-compliance was due to confusion about what the organisation expected (i.e. goal differences), would not be the optimal solution for non-compliance arising due to poor task design making compliance extremely difficult. A user of the tool from a chemical manufacturing plant noted that the generic approach allowed him to identify a non-compliance as unintentional, and that it would be most effectively addressed through a behavioural approach, whereas his original perception was that it was due to deliberate deviation. It is also important to emphasise that this is a solution set – it’s not a set of causes. One powerful cause is a misunderstanding of the demands of the prevailing situation, leading to incorrect rule selection. This can arise for many reasons, including
150
J. Berman, P. Ackroyd & P. Leach
poor situation awareness (e.g. Banbury and Tremblay, 2004), or the effects of recognition-primed decision-making where the decision-maker uses probabilistic reasoning based on past experience (e.g. Klein, 1993). The importance of the solution-set is that it provides potential means of overcoming the consequences of these effects. However, other approaches to improved management of situation awareness and decision-making remain valuable, and this underlines the importance of a coherent approach to compliance management. Additionally, whilst improving the rule may improve compliance, it is also important to consider why the rule was not optimal. It is inevitable that there will be ‘bad’ rules – systems and processes change and evolve. The challenge arises when the organisation makes it difficult for a rule to be improved. This will tend to lead to non-compliance rather than identification and improvement. Thus, a proactive solution will focus also on improving the change management arrangements. It may further wish to consider means of proactive review of procedures to identify challenges (such as might arise through a HAZOP-style evaluation process). In one instance, an organisation was concerned about staff failure to wear highvisibility vests. This arose in part due to inconvenience and was eventually overcome by making the overalls high-visibility and hence removing the need for the vests. By making compliance easier, the need for the ‘rule’ was removed.
What would help? A number of challenges have been noted in terms of recognising and understanding the nature of a given compliance problem, the ‘contract’ between the organisation and the individual and how that contract affects behaviours and goals, the range of potential solutions (which span the gamut of ergonomics), only a subset of which will be relevant to a given compliance problem, and understanding how to implement them. These challenges are no different from those facing any ergonomics intervention and should be amenable to a similar ergonomics approach: • • • •
Recognise the problem (be clear about what is happening) Analyse the problem (understand the activity, constraints and goals) seeing the solution (considering a set of potential solutions) devising an effective implementation strategy (it is likely to involve interventions that affect attitudes and behaviours)
Guidance tends to be beneficial in supporting selection, tailoring, implementation and monitoring of solutions. A further requirement is to help the organisation not only to be able to correct identified shortfalls that could lead to non-compliance, but also to become more resilient to the influence of those factors. In other words, the optimal approach would reduce the likelihood of future shortfalls arising. The rail industry software tool (op. cit) was developed to support organisations that wished to understand and manage safety critical rule compliance. Unsurprisingly, the tool itself is generic (although the examples are predominantly drawn from the rail industry). In developing the tool, much time was spent working with a range of rail industry organisations (Train Operating Companies, infrastructure
. . . It’s the way that you do it: Ensuring rule compliance
151
maintenance companies, etc) to understand the range of contexts and potential users of the tool, and the questions the tool was to support. What emerged was clarity of the elements that could give rise to non-compliance and the importance of identifying which one(s) were relevant in a given context (and typically there would be more than one, and therefore also more than one preferred solution). It was also apparent that the tool was likely to be of most use to those managers and supervisors who were best placed to address the issues. The tool was constructed as a set of questions that would encourage the user to explore the evidence for potential factors that could affect compliance, and then to consider various means of addressing those factors. The tool linked the causes and potential solutions, together with examples of how such solutions might be implemented. This was also found to be a key element of the tool, as it is important that the set of selected solutions are tailored to the specific demands of the organisation. In other words, the tool assisted users to identify their own solutions, rather than presenting ‘off-the-shelf ’ solutions. It provides examples. The tool was empirically derived, and its validation has tended to be pragmatic – organisations have found it has led them towards solutions that might not have been immediately apparent to them, but which appear to have led to reduced noncompliance. In piloting the tool, it was clear that there was a need to be able to communicate with senior management concerning the nature of the issues and the potential ‘solution space’ in order that appropriate resources would be made available. However, it was also clear that selection of appropriate solutions required cooperative effort between managers and staff at all levels, and the tool needed to be able to facilitate this communication. It is important not to imply a single solution. The questions arising were also important. Whereas many managers might initially make use of the tool to allow them to understand and resolve non-compliances identified through incident reporting and investigation, it became apparent that this was often merely the means of engaging with those managers, and that once familiar with the approach they were equally interested in proactive assessment of potential non-compliances – perhaps arising from planned changes to working practices. This prompted the development of a specific part of the tool that supported proactive assessments.
Integrating solutions Implementing effective solutions requires a coherent and wide-ranging approach. It’s not sufficient simply to address a particular instance of non-compliance – nor to focus on one aspect, such as poor task design, or poor procedures. There are two distinct areas where solutions may lie, and typically only one is considered. Depending on the nature of the non-compliance, there will be a tendency to look for solutions either within conventional ergonomics (task design, interfaces, procedures, training, workload, etc), or within organisational factors (supervision, culture, leadership, communications, sanctions and rewards, etc). It is not our intention to imply that ergonomics excludes organisational factors – ergonomics can be considered to embrace the latter – the distinction is drawn to highlight the perception
152
J. Berman, P. Ackroyd & P. Leach
that can exist within organisations. All too often there is an attempt to resolve non-compliance through improved procedures or training, without considering the importance of addressing the organisational factors. Attention to conventional ergonomics is necessary but not sufficient. Effective management of compliance requires a balance of both of these aspects – it is rarely the case that an organisation has deficiencies in only one of these areas. This returns to the issue of the ‘contract’ between staff and organisation. The organisation has an obligation to ensure that the ergonomics issues are addressed such that compliance is made as easy as possible, and non-compliance is made both difficult and non-rewarding. This permits the member of staff to fulfil their part of the contract which is to operate systems in a compliant manner. At the same time, the organisation also has an obligation to support the member of staff through provision of effective supervision etc., which in turn can reveal goal disparities. Without a clear and recognised balance between the two elements, there is the potential for the solutions to fail. Furthermore, it is attention to the organisational issues that will provide both a clear demonstration to staff that non-compliance is unacceptable to the organisation (and that it is prepared to commit resources to reducing it), and also a clear demonstration that the organisation is not merely paying lip service to improving compliance but instead is establishing processes that will provide high quality information on which to act. Senior staff and line managers create the perceived priorities within an organisation and, often unwittingly, place an undue priority on commercial or operational goals thereby undermining safety. Staff may perceive that it is acceptable, or even required, to take risks to achieve the company’s goals. Language, allocation of resource and what managers show interest in can all create this position. Staff may feel that it is not appropriate or possible to raise safety concerns. This is an area where senior managers need to consider their own behaviours carefully. It is vital that they connect directly with the workforce to find out what is really happening, what concerns staff have, and to consider the nature of the culture within the organisation. Positive environments where success is celebrated can inadvertently stifle concerns. Senior managers either not wanting to listen to ‘bad news’ or appearing to challenge such news aggressively will inevitably prevent key information being raised and acted on. Middle managers become a filter that either blocks or distorts information from the workforce. Good news is quickly passed upwards whilst bad news may be reduced, downplayed or blocked. Thus, effective management of compliance requires attention to attitudes and culture issues, in order to maximise the benefits from attention to more conventional ergonomics issues such as procedures, training and task design.
Resilience No discussion of compliance would be complete without considering organisational resilience. It’s important that organisations not only address the current factors that
. . . It’s the way that you do it: Ensuring rule compliance
153
might encourage non-compliance, but that they also ensure they are able to prevent the recurrence of those or similar factors, or the gradual erosion of the effectiveness of controls. Many organisations rely on procedural controls to ensure effective management arrangements – and these more than any are frequently at risk of degradation due to non-compliance. Many events from finance (Société Générale, Barings Bank) or conventional high-hazard industries (Longford gas explosion, Royal Commission, 1999; Grayrigg train derailment, RAIB, 2008) have the failure of procedural controls as a significant causal factor. Amalberti (2006) presents a revision of Reason’s resilience model as a cascade of resiliences, where a series of sets of prevention, detection, recovery and mitigation are provided to prevent failures from propagating into accidents. Control of compliance should be considered in the same manner, whereby arrangements are implemented and maintained to prevent not only the identified non-compliances, but also potential future or undetected ones. It is the ability to develop arrangements that remain robust over time that is of particular importance, and why this paper has emphasised goals and solution selection rather than the solutions themselves. It is also important to consider how those arrangements can degrade over time precisely because of the factors that give rise to non-compliance. This is noted by the present authors when considering organisational drift (Berman and Ackroyd, 2006). Attention to improving compliance is a key defence against drift, and a key factor in supporting resilience. Furthermore, the defences against non-compliance are themselves powerful means of enhancing resilience. It was noted above that the proactive assessment element of the toolkit was seen by users to be of significant value, such as for the assessment of planned changes. This is particularly relevant to organisational drift and resilience. One of the challenges with respect to drift is compensating behaviours – where staff adopt a ‘can-do’ attitude to overcome emerging challenges. The difficulty arising from this is that those compensating behaviours can mask the potential degradation in resilience. They are frequently indicated by deviations from agreed procedures, but by virtue of their nature are often not immediately apparent to the organisation. An example we have seen is where a change to a procedure gave rise to resource constraints that meant a particular task check became difficult to undertake. However, because the check was of a high-reliability system and had never revealed a fault, staff were not concerned that it was no longer undertaken. But the informal nature of this compensation meant that the organisation still believed that the check was being made, and claimed benefit from it in their safety assessment.
The way forward Our experience is that successful compliance management is a multi-faceted process. In particular, it requires both a proactive and reactive approach to identifying potential non-compliance, and then a balanced programme of solutions driven out from an understanding of goals, that cover both the design of tasks (in the broadest
154
J. Berman, P. Ackroyd & P. Leach
sense) and how the organisation goes about its business. The key elements of compliance management appear to be: • • • • • • • • • •
Understanding individual goals Identification of clear and realistic goals, targets and benchmarks Focus on competence and supervision Consideration of incentives and sanctions (both intentional and unintentional) Provision of resources and equipment (including all aspects of task and job design) Application of consistent standards and procedures Effective investigation of concerns, successes, incidents and near-misses Effective arrangements for workforce involvement and change management Provision of effective review Accountability
There is no single ‘best’ method for improving rule compliance. However, the rewards of doing so are significant, in terms of improving organisational resilience and performance. Management of rule compliance is central to the domain of ergonomics, but must start from an understanding of the disparity between the goals of the individual and the organisation, which are then impacted by organisational issues, workload and ergonomics. By employing competent people, it is inevitable that there will be constant pressure towards non-compliance (albeit without malicious intent). This therefore requires constant attention to the management of non-compliance if the organisation is to survive. Most staff comply with arrangements if they can recognise the value and importance of doing so. All too often that value and importance is inadvertently hidden. Ergonomics is central to compliance management – but only if a broad perspective is maintained.
References Amalberti, R. 2006, Optimum System Safety and Optimum System Resilience. In: E. Hollnagel, D. Woods and N. Leveson (eds) Resilience Engineering. Aldershot: Ashgate Banbury, S. and Tremblay, S. 2004, Situation Awareness: A Cognitive Approach. Aldershot: Ashgate Berman, J. and Ackroyd, P. 2006, Organisational Drift – a challenge for enduring safety performance. Proceedings of Hazards XIX, Rugby: IChemE Berman, J., Ackroyd, P., Mills, A. and Davies, T. 2005, Management Toolkits – Solutions for Rule Compliance. In: J. Wilson, etc (eds). People and Rail Systems: Human Factors at the Heart of the Railway. Aldershot: Ashgate HFRG, 1995, Improving Compliance with Safety Procedures. London: HSE Klein, G. 1993 A recognition-primed decision (RPD) model of rapid decision making. In: G. Klein, J. Orasanu, R. Calderwood and C. Zsambok (eds) Decision Making in Action. New York: Ablex
. . . It’s the way that you do it: Ensuring rule compliance
155
McDonald, N. 2006, Organisational Resilience and Industrial Risk. In: E. Hollnagel, D. Woods and N. Leveson (eds) Resilience Engineering. Aldershot: Ashgate. RAIB, 2008, Derailment at Grayrigg 23 Feb 2007. RAIB Report 20/2008. Dept for Transport Royal Commission, 1999, The Esso Longford Gas Plant Accident: Report of the Longford Royal Commission, State of Victoria
HEALTH ERGONOMICS AND PATIENT SAFETY
PUBLIC REQUIREMENTS FOR PATIENT-HELD RECORDS J. Binnersley1 , A. Woodcock1 , P. Kyriacou2 & L.M. Wallace1 1
Coventry University, UK 2 City University, UK
Electronic patient-held record devices enable people to carry around information about their health. They need to be easily accessed in emergency situations and should provide enough information to enable front line staff to deliver effective care. Although it has been suggested that patients taking responsibility for their records could improve patient safety, little is known about attitudes towards this. In taking a user centred approach to the development of such a system, surveys were conducted with the public and health professionals. The survey of 258 members of the public showed that 85% thought patient-held records would be useful. The types of information that most people wanted to carry were name, allergies, long term conditions, current medication and blood group. A smart card design was preferred by 62% of the participants and 68% thought it should be provided free of charge.
Introduction In many areas of health care, professionals rely on patients or their relatives to provide information about medical history. Accurate information is vital for the safety and effectiveness of clinical decision making but many people struggle to provide this (for example because they are unconscious, have forgotten, or do not know or understand what information to provide). Although it has been suggested that patients taking responsibility for their records could improve patient safety (Hall, 2007), little is known about the UK public’s attitudes towards this or what the design and requirements of such a system would be. The study reported here is part of a wider Engineering and Physical Sciences Research Council funded project with City University to develop a patient held health record device which takes into account the requirements of the public and health care professionals. It will act independently of health service initiatives, being kept, carried and updated by the patient. This paper focuses on the first stage of the project, a survey undertaken to understand attitudes towards, and requirements for, such a device. The information will be used by product and software designers to create solutions which meet user requirements and acceptance criteria. Previous studies, such as Ross et al. (2005) have pointed to the interest of patients in having certain types of information available for privileged access. However, Pyper et al. (2004) highlighted concerns that unauthorised people might gain access 159
160
J. Binnersley et al.
to electronic records. These, coupled with worries about surveillance might mean that regardless of the benefits of such a system, it may not be accepted by the UK public. Additionally, there has been a lack of consensus about the types of information that should be entered, access rights, the design of the device, and input and display requirements. Electronic patient held record systems are currently used in many countries and many of these use smart cards. They are used as combined ID/health cards in Malaysia and Germany (Frontier Strategies, 2001). In the USA, devices being used include USB or memory sticks such as MEDeCARD and dog tags for the military (Burnham, 2000). In the UK a pilot study was conducted between 1989–1992 in Exmouth, where13,000 smart cards were used by patients (NHS Management Executive, 1990). Patients were able to restrict the information contained on the cards, but provision was made to include health care professional identifiers, ethnic group, language spoken, religion, medication details, allergies/drug sensitivities, donor status, emergency information, symptoms, examination findings, laboratory information, tests and measurements, diagnostic information, employment details and administrative data. Access to the information was available to health professionals. The main result of the trial was that the patients liked the concept of an electronic patient-held card and also suggested that more information should be included on the cards. The study was not continued, with the availability of technology at the time limiting its wider feasibility. The survey outlined below was undertaken with the aim of collecting information from the UK public about support for patient-held records, potential barriers to their use, the sort of information that should be held on a patient held health record, access rights and the user requirements for a device.
Method In order to determine the questions to include in the survey five focus groups were held, consisting of a total of twenty-five people. These were members of the public, designers, IT specialists, health professionals and people with chronic illnesses. The full survey was conducted in pharmacies in London and the Midlands. Pharmacies were selected to include participants from a range of ethnic backgrounds, geographical locations and social classes. Customers over the age of 16 were invited to participate in the study and were given verbal and written information. Only those who signed the consent form were included. Some participants with poor eyesight and levels of English literacy were assisted in completing the questionnaire and 258 questionnaires were returned.
Results Participant characteristics The age range of the participants was 17–89 years, with a mean age of 45 years. 43% of the participants were male, and approximately two thirds were of white
Public requirements for patient-held records
161
European origin. The largest occupational group was white collar professionals (27%). 61% lived in the Midlands and 39% in London. 26% considered themselves to have a long term health problem and 13% had already experienced being given part of their health records to look after. The most common form of these had been paper records, including letters and repeat prescriptions. The most commonly used paper records were maternity records and the most frequently cited advantage of keeping records was being able to share information with health professionals, stated by eight participants. The most frequently listed disadvantage was the possibility of losing them, mentioned by six participants.
Support for patient-held records 85% of the participants said they would find a patient-held record device useful. The most common circumstance where participants thought a patient-held record would be useful was if they were too ill to give information to a health professional. The most common concern was that unauthorised people would gain access to the information and 64% of the participants were worried about this. Participants were provided with a list of items and asked whether they thought these should be included on a patient-held device. The answers given in the table below represent positive responses for each item. The results show that the items most participants thought should be on patientheld records were current medication, name, allergies, blood group and long-term conditions. Few participants could think of other items that they would want included but nine specifically stated that they would not want their address on a device. Most participants thought that all health professionals should be able to see these items. 81% of the participants stated they agreed or strongly agreed that health professionals should be able to access different levels of information according to their profession. Seven participants specifically stated that they thought pharmacists should be able to access some information and 27 participants stated that non-medical personnel should not be able to see the information.
Table 1.
Information to be included on a patient-held record device.
Item Current medication Name Allergies / intolerances Blood Group Long-term conditions Next of kin Age Previous major health problems NHS number Address
Percentage agreement 92 92 92 91 89 88 86 75 75 69
Item Disabilities Dietary information Living will and donor information Implants Carer contact Religion Information about social services care Non-prescribed medication Ethnicity
Percentage agreement 63 59 55 54 50 42 39 34 34
162
J. Binnersley et al. 70 60 50 %
40 30 20 10 0 Smart card Key fob
Jewellery
Others
Device Computer memory linked to a mobile stick phone
Figure 1. Answers to question about preferred form of device.
When asked to indicate whether they agreed or disagreed with statements regarding their attitudes towards patient-held record devices, the majority of participants thought it would be good to use a patient-held record device (77%), that people who were important to them would approve of this behaviour (64%), that they would find such a device easy to use (82%) and would intend to use one if such devices were available (76%). Most participants wanted themselves or their doctor to be able to update the information. The majority (85%) wanted to be able to update information at their doctor‘s surgery and to use a keyboard form of entry (71%). When asked what type of device they would be most likely to use, the most popular was smart card, ticked by 62% of the participants who answered the question. Figure one illustrates the device preferences. When asked how much they would be willing to pay for such a device, 157 (68%) of the participants indicated that they thought it should be provided free of charge. Only seven people were willing to pay over £20.
Discussion The present survey replicates the findings of Ross (2005) who found that shared medical records were almost universally endorsed across a broad range of ethnic and socioeconomic groups. Pyper et al. found a high level of support for electronic records, but also concerns about unauthorised people gaining access to the data. Designers will need to consider using personal access codes in order to keep the data as secure as possible. They will need to address what happens when the card is stolen, misplaced or inaccurate information is entered and the consequences of this. Most members of the public would like a wide range of information held on the device, with the most commonly cited being current medication, name, allergies,
Public requirements for patient-held records
163
blood group and long term conditions. The NHS Management Executive study (1990) found that patients tended to want more information to be included after they had become used to having the device. However, this may not be the case if the onus falls on the patient to enter and regularly update such information. This will be a major consideration in the design stage, as ease of use will significantly affect regularity of use. Liaw’s (1993) study found that most participants preferred a credit card or walletsized patient-held record device. Similarly, in the current study, most participants preferred a smart card design. Advantages of this over other design forms are portability, familiarity and cost. A ‘card’ format may also be more acceptable as an input device (e.g. as opposed to a piece of jewellery) as it can be accommodated more readily in existing readers. The card itself can also be a carrier of information (e.g. a picture of the patient, life threatening illnesses and allergies). The design and the relationship between this and the information carried in the software is being addressed through development of a prototype. The survey had several limitations. Some potential participants declined to take part because they did not see the advantage of such a system for them or because English was not their native language. The study did not include consideration of how patient-held records could be managed for children or adults who would be unable to take responsibility for their own records. None of the participants in this study were under the age of sixteen. It is likely that these groups may derive greater benefit from a patient-held record system. Lastly, some participants found it difficult to imagine such a device (and system) without seeing a prototype. Future work could therefore address increasing the public‘s confidence in the security of databases and awareness of the benefits of patient held health records. One way of achieving this will be through the development of working prototypes which will enable members of the public to achieve a greater understanding of the costs and benefits of such a system. For example, costs in terms of data entry, remembering to carry the card and update it. The team are also mindful that only patients have been considered. Without the acceptance and use by health care professionals, a patient held health records system will be redundant. Therefore a similar survey is currently being conducted with this group.
Acknowledgements The research has been supported by an Engineering and Physical Sciences Research Council grant number EP/F00323411 28/02/2007. Lloydspharmacy permitted data collection on their premises.
References Frontier Strategies, 2001, Smart cards for smart customers (Frontier Strategies, Canada) [online] Burnham, T. 2000, Dog Tag Health Card; Soldiers Dec 2000
164
J. Binnersley et al.
Hall, J. 2007, What roles for the patient in patient safety research? In conference proceedings Patient Safety Research: Shaping the European Agenda, 24–26 September 2007, Porto, Portugal [online] Liaw, S.T. 1993, Patient and general practitioner perceptions of patient-held records. Family Practice 10, (4) 406–415 NHS Management Executive. 1990, The Care Card: Evaluation of the Exmouth project. (HMSO, London) Pyper, C., Amery, J., Watson, M. and Crook, C. 2004, Patients‘ experiences when accessing their on-line electronic patient records in primary care. British Journal of General Practice 54, (498) 38–43 Ross, S.E., Todd, J., Moore, L.A., Wittevrongel, L. and Lin, C.T. 2005, Expectations of patients and physicians regarding patient-accessible medical records. Journal of Medical Internet Resources 7, (2) 1–32
SYSTEMS ANALYSIS FOR INFECTION CONTROL IN ACUTE HOSPITALS P.E. Waterson Department of Human Sciences, Loughborough University, Loughborough, LE11 3TU, UK This paper makes the case for applying a systems perspective to the analysis of hospital-based infection outbreaks. Most of the research that has been conducted on behavioural aspects of infection control has focused on explanations at an individual level of analysis (e.g., interventions to improve hand washing). The infections outbreaks at the Maidstone and Tunbridge Wells NHS Trust are analysed in detail using an established framework for risk management. The paper further outlines the human and organisational issues raised by the analysis and provides a means through which these aspects of infection can be highlighted as part of a future research agenda within systems ergonomics.
Introduction Within the last few years the subject of hospital infection control has become the subject of much media attention (e.g., BBC Panorama, 2008). A number of high profile hospital outbreaks within the UK involving bacterium such as Clostridium difficile (C. diff.) and MRSA (Methicillin-resistant Staphylococcus aureus) and the number of mortalities resulting from these outbreaks, has made infection control into a central priority for the UK NHS and other health care systems worldwide (Allegranzi et al., 2007). Much of the debate so far has concentrated on improving hygiene within hospitals (e.g., hand washing), very little research has been conducted on the wider behavioural, social and organisational factors that may also determine infection control outbreaks (Griffiths, Renz and Rafferty, 2008). The intention of the present study is to consider the potential of adopting a systems ergonomic perspective towards hospital-based infections. More specifically, the paper describes the advantages to be gained from applying existing systems analysis techniques to infection outbreaks and using these to draw out both lessons learnt as well as strategies for improvement. In order to demonstrate the value of such an approach the paper focuses on a specific case study namely, the Maidstone and Tunbridge Wells NHS Trust outbreaks which occurred between 2005–2007 (Healthcare Commission, 2007). 165
166
P.E. Waterson
Systems analysis and systems ergonomics Hospitals represent a good example of a complex large-scale sociotechnical system involving a large diversity of professions spanning a range of roles and specialisms as well as technologies that range from the latest eHealth applications (e.g., electronic records) to more established aspects such as physical design components (e.g., wards and buildings). Within systems ergonomics, Rasmussen (1997) has provided a modelling framework for understanding the dynamic interaction between levels within large-scale sociotechnical systems. Vicente and Christoffersen (2006) have used the modelling framework to identify the lessons learnt from the May 2000 outbreak of E. coli which occurred in Walkerton, Canada. Their analysis used a graphical representation of the contributing factors that led up to the Walkerton outbreak. These ranged from decisions made at governmental levels (e.g., privatisation initiatives), the action of actors within the system (e.g., failures to take water samples), as well as equipment failures (e.g., shallow water wells). In the next section the main events leading up to the outbreaks at Maidstone and Tunbridge Wells are outlined. These are used to suggest ways in which the outbreak could be analysed in more detail, and modelled using the Rasmussen and Vicente frameworks.
The Clostridium difficile outbreaks in the Maidstone and Tunbridge Wells NHS Trust Background to the outbreaks and timeline During the period between April 2004 and September 2006 an estimated 90 people died at the Maidstone and Tunbridge Wells NHS Trust as a result of becoming infected with the Clostridium difficile (C. diff.) bacteria (Healthcare Commission, 2007, p. 5). C. diff. is the major cause of serious bacterial infectious diarrhoea acquired in hospitals in the UK and is particularly resistant to drying, chemical disinfectants and alcohol. The main events at the Maidstone and Tunbridge Wells NHS Trust are summarised in table 1.
Contributory factors leading up to the outbreaks The Healthcare Commission (2007, HC, 2007) report identified a number of factors that contributed to the outbreaks that occurred with the Trust. These can be summarised in terms of five main themes: the role played by external organisations; management of the trust; clinical management on the hospital wards; the role played by the infection control team; and, equipment and hygiene factors.
The role of external organisations Within the report both the setting of government-led targets and financial pressures on NHS Trusts are mentioned as background, contributory factors that had an impact on the day-to-day operation of the Maidstone and Tunbridge Wells Trust. In particular, the report mentions the need for Trust board members and managers
Systems analysis for infection control in acute hospitals
Table 1. Time period
167
Summary timeline of infection outbreaks.
Event
April 2000 2001/2–2005 October 2005– September 2006 Before 2005
Trust established following merger between two other local NHS Trusts High turnover of senior managers and period of organizational stability More than 500 patients developed the infection, 60 patients estimated to have died due to C. difficile infection. Trust has a high level of infection with C. difficile but no one in the trust or local health authority was aware of this Autumn 2005 Number of patients infected doubles. Approximately 150 patients affected, a number of whom died as a result of the infection. (first outbreak) April–Sept. 2006 258 patients in total affected Beginning 2006 Number of newly infected patients declines. April 2006 Trust recognizes it has a major outbreak and reports this to strategic health authority and health protection unit. (second outbreak) April 2007 Healthcare Commission finds unacceptable examples of the use of contaminated equipment October 2007 Healthcare Commission Report published.
to meet targets for the use of beds. Higher bed occupancy meant that there was less time for the cleaning and a higher probability of transmission of infection between patients (HC, 2007, pp. 69–70). The need to meet financial targets such as spending on equipment and buildings also placed pressure on the Trust to cut back in areas that impacted upon infection control such as financing for new buildings and isolation areas. Infection control within the UK NHS is regulated by a number of bodies including the Health Protection Agency (HPA). The remit of the HPA is to provide advice and support to NHS, local authorities and other agencies with regard to public health issues. The creation of the HPA in April 2005 coincided with the first outbreak at the Trust. One part of the HPA, the health protection unit (HPU), was set up in order to support organisations in their management of infections. The report highlights that this caused some confusion within the Trust at the time of the outbreaks, as the expectation was that the HPU could give provide guidance covering the supervision and monitoring of infection control. The HPU did not have close involvement with the Trust and generally worked in a reactive way, responding to concerns as they arose (HC, 2007, p. 8). Similar problems were encountered within the much larger Strategic Health Authority (SHA) who are responsible for implementing government policy and fiscal control within regions of the UK.
Management of the Trust The report describes a catalogue of problems and failures associated with the management of the trust at the time of the outbreaks. In terms of clinical risks and incidents, management strategy in general “had been fragmentary and poorly understood” (HC, 2007, p. 77). The reports from an internal group set up within
168
P.E. Waterson
the Trust in order to analyse complaints, claims and incidents highlight, amongst others, the following issues: the unsatisfactory nature of some “escalation” areas (areas temporarily set up to deal with infected patients); the impact that the accident and emergency (A&E) target had on the quality of care; poor quality handover and transfer to wards from A&E; concerns about staffing levels, and, bank staff managing wards on some shifts. The style of leadership within the Trust and the overall management culture are criticised in the report. Many staff described the leadership of the chief executive as being “autocratic” or “dictatorial” (HC, 2007 p. 91). The report concluded that the person appointed as director of infection prevention and control had “no real understanding of the role at the outset” (HC, 2007, p. 5). Turnover of managers and directors was also high. Finally, the trust’s management of staffing is criticised heavily within the report in several places. The number of nurses working on wards had fallen since the period 2002/03 and at the same time the number of beds had also reduced. In 2006/07 the number of nurse per bed was 1.52, the same number as in 2003/04 (HC, 2007, p. 82). Trust managers had not carried out a comprehensive review of staffing levels or a determination of minimum staffing levels.
Clinical management on the hospital wards A review of the case notes of 50 patients who had died having had C. diff. found that in 80% of the cases, at least one element of the clinical management, or monitoring of C. diff at ward level was unsatisfactory (HC, 2007, p. 4). A number of elements were mentioned, including: infrequent reviews of patients by doctors; lack of systematic monitoring as to whether or not a patient was recovering from C. diff.; and, failure to change antibiotic treatment when a patient failed to respond to the initial treatment (HC, 2007, p. 4). Delays in starting treatment occurred on the wards, mostly because there was a delay in sending samples for analysis (HC, 1007, p. 33). There was also little evidence that once C. diff. had been diagnosed, that patients were monitored for severe signs of the infection (HC, 2007, p. 34). In other cases, it was clear that diagnoses were either not considered or had been missed. In 34% of the cases reviewed, medical records did not indicate that a regular review of C. diff had taken place (HC, 2007, p. 38). The management of fluids and nutrition on the wards was also inconsistent. In 36% of the cases there was evidence of poor fluid management and in 34% nutritional needs had not been assessed or managed (HC, 2007, p. 38).
The infection control team The role played by the infection control team within the trust was a complex one and one made difficult by problems relating to accountability, the amount of resources available to them and their ability to function as a team. The arrangements for accountability were not clear (HC, 2007, p. 54) and it was not clear who was responsible for the team. Infection control nurses were accountable to the director of nursing, however, the pathology manager held the budget for these nurses, but did not consider that he had any management responsibly for infection control. Not
Systems analysis for infection control in acute hospitals
169
until September 2006 did the trust take steps to clarify the management of the team (HC, 2007, p. 51).
Equipment and hygiene Hygiene practices within the trust and the state of hospital buildings contributed a great deal to the outbreaks. Wards, bathrooms and commodes were not clean and patients had in some cases to share equipment (e.g., Zimmer frames) which were not cleaned before use (HC, 2007, p. 4). The infection control team were keen to isolate patients once they had been identified as C. diff. cases, however the scarcity of side rooms made this difficult. As a result many patients before and after the outbreaks were kept on open wards. The design of buildings and their age meant that many wards did not have sufficient space for storage or the provision of hand basins in utility rooms. The buildings in the trust were generally old or in a poor state of repair and when they were first opened did not have adequate cleaning and laundry services (HC, 2007, p. 6).
Analysing the outbreaks: A systems perspective The outbreaks which occurred within the Maidstone and Tunbridge Wells Trust represent the combined impact of a complex set of factors extending over several years. In common with most examples of accidents, disasters or large-scale adverse events, the outbreaks are best interpreted as arising through the combination of a number of interrelated systemic factors and influences (Turner, 1978; Reason, 1995). Figure 1 attempts to use some of the elements of Rasmussen’s (1997) risk management framework in order to further analyse the outbreaks. In order to illustrate the framework as it applies to the outbreaks, a small sample of the contributory factors are used to link together some of the system components. In the following sections, the various levels and the boundaries between them, are examined in more detail.
Government, regulatory bodies and trust governance At the very highest level of the system it is difficult to isolate the role played by government-set targets as a contributory factor leading to the outbreaks. Targets placed many individuals, particularly those at trust board and management levels under a great deal of pressure. This pressure in itself may have led them to make poor decisions, and in some cases to prioritise bed occupancy rates at the expense of the risk of an infection outbreak. Previous research on the influence that targets have on management decision-making in health care tends to be equivocal. Bean and Hood (2006) for example, show that the impact of satisfying a specific target (e.g., hospital waiting times) has not been analysed in terms of how this influences other related services (e.g., quality of care). Others have suggested that health care targets represent: “tin openers rather than dials … they do not give answers but prompt further investigation and inquiry, and by themselves provide an incomplete and inaccurate answer” (Carter, Klein and Day, 1995). Within the trust it is likely
170
Figure 1.
P.E. Waterson
Mapping of infection outbreak influencing factors with system levels and boundaries.
that targets exerted considerable pressure on the system as a whole and this pressure filtered down various levels of the system. It is possible that the drive to comply with these targets increased the likelihood of an adverse event or set of events taking place at some stage within the trust. Poor communication, confusion of responsibilities and accountabilities between and within the various regulatory bodies delayed the time in which they could react to the outbreaks. A separate report by the Healthcare Commission (2008) examined the underlying causes of serious failures in NHS health care providers and identified large-scale organisational processes such as mergers and poor change management procedures as common factors. Within the wider literature on disasters (e.g., Perrow, 1999) the nature of organisational linkages and structures are also widely acknowledged to be significant explanatory factors.
Hospital management Within the hospital the actions of senior managers were identified as significantly contributing to the failure to prevent and deal with the outbreaks. The link between management, human resource management (HRM) practices and work performance outcomes has been investigated in detail in the last few years. Wood and Wall
Systems analysis for infection control in acute hospitals
171
(2002) for example, reviewed the evidence that suggests there is a link between highinvolvement HRM practices and employee productivity. High involvement HRM practices typically include empowering employees to make their own decisions and the presence of self-managed teams. The review showed that there these types of practices in organisations do tend to increase levels of employee productivity. Similar effects have been shown between HRM practices and measurements of safety outcomes (e.g., number of adverse events). In general, there is strong evidence to suggest that aspects of management behaviour partially shape and determine the culture of safety within organisations (e.g., Zohar, 2000). Within health care specifically, West et al. (2002) carried out a large-scale survey of the relationship between HRM practices and general in-hospital mortality. The survey showed some aspect of high involvement HRM were associated with lower mortality rates after adjustment for patient and hospital characteristics. Aside from the way in which senior managers behaved at the trust, the questions still remains as to why they ignored, or at least failed to realise the seriousness of the outbreaks and their consequences. Many of the managers interviewed in the original Healthcare Commission report reported that they were aware of how serious the situation had become within the trust, but were powerless to do anything about it. One possible explanation is what Vaughan (1996) in her study of the Challenger shuttle disaster termed the “normalization of deviance”, namely that managers over time began to accept and take for granted the level of infection risk within the Trust. Only after the level of risk built up to a point where it could not be controlled, did they begin to realise the gravity of the situation.
Clinical management and equipment and buildings Understaffing and general lack of resources together played a part in the outbreaks. Staffing ratios and levels of staff morale almost certainly contributed to the problem of containing the spread of infection on the wards. In general, the research literature provides some evidence that lower levels of staffing increase the likelihood of infections occurring. Hugonnet et al. (2004) (cited in Griffiths et al., 2008) examined the numbers of nursing staff and staff downsizing relative to infection levels. The researchers found an inverse relationship between staff downsizing and the rate of hospital-based infection. Curiously, little research has been conducted on the impact of job satisfaction/morale on hospital infection levels, however, work in other domains (e.g., manufacturing and service industries) suggests that lower levels of satisfaction are clearly linked to lower levels of job performance (e.g., Parker, 2007). It might be conjectured that the behaviour of clinicians and other health care professionals within the trust shares similarities with those of senior managers and trust board managers. Many individuals at ward level were aware of the levels of poor hygiene and inadequate patient monitoring practices, but saw no way to improve the situation. Weick and Sutcliffe (2003) analysed data from the Bristol Royal Infirmary Report (2001) and concluded that hospital staff became locked into particular lines of action or behaviour where they “search for confirmation that they are doing what they should be doing” (p. 73). These so-called “cultures of
172
P.E. Waterson
entrapment” inhibit an organisation’s ability to break out of patterns of behaviour that over time can lead to adverse outcomes. In the case of the trust they may provide some means with which to explain shared boundary spanning behaviours between levels within the hospital subsystem (figure 1).
Ways forward and conclusions The analysis presented in the paper has shown that there are advantages in analysing hospital-based infection outbreaks from a system’s perspective. Many of the issues that have been discussed have not been researched within the patient safety literature in much depth, particularly organisational phenomena (Waterson, 2008). It is still the case that most research within the area of infection control has concentrated on individual levels of analysis (e.g., behavioural interventions in the form of hand washing campaigns). The paper has only touched upon some of the behavioural issues involved with a system as complex as hospital-based healthcare. Future research should examine causal links between system levels in a more systematic manner and address the issue of influences at the meso-level of analysis, that is linkages and relationships between individual (macro-) and organisational (macro-) system levels (Karsh and Brown, in press). Further empirical studies which focus on behavioural aspects of infection control which go beyond individual levels of analysis are needed. In addition, there appears some scope for modelling the relationships between system components and comparing these models against hospital-based data (e.g., Brailsford and Schmidt, 2003).
References Allegranzi, B. Storr, J., Dziekan, G., Leotsakos, A., Donaldson, L. and Pittet, D. 2007, The first global patient safety challenge “Clean Care is Safer Care” from launch to current progress and achievements. Journal of Hospital Infection, 65 (supplement 2): 15. Bean, G. and Hood, C. 2006, Have targets improved performance in the English NHS? British Medical Journal, 32, 419–422. BBC Panorama 2008, How Safe is Your Hospital? (Broadcast on 27th April, 2008), (http://news.bbc.co.uk/1/hi/programmes/panorama/default.stm, accessed 28th April, 2008) Brailsford, S. and Schmidt, B. 2003, Towards incorporating human behaviour in models of health care systems: An approach using discrete event simulation. European Journal of Operational Research, 150, 19–31. Carter, M., Klein, R. and Day, P. 1995, How Organisations Measure Success: The use of Performance Indicators in Government. (London: Routledge). Department of Health 2001, Learning from Bristol. The Report of the Public Inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984–1995. (London: HMSO). Griffiths, P., Renz, A. and Rafferty, A.M. 2008, The impact of organisation and management factors on infection control in hospitals: a scoping
Systems analysis for infection control in acute hospitals
173
review. Kings College, London. (http://www.kcl.ac.uk/content/1/c6/04/08/39/ Infectioncontrolreport.pdf, accessed 15th September, 2008). Healthcare Commission 2007, Investigation into outbreaks of Clostridium difficile at Maidstone and Tunbridge Wells NHS Trust. (http://www. healthcarecommission. org. uk/_db/_documents/Maidstone_and_Tunbridge_ Wells_investigation_report_ Oct_2007.pdf (accessed 8th April 2008). Healthcare Commission 2008, Learning from Investigations (http://www. healthcarecommission.org.uk/_db/_documents/Learning_from_investigations. pdf (accessed 17th September 2008). Huggonet, S., Harbath, S., Sax, H., Duncan, R.A. and Pitter, D. 2004, Nursing resources: a major determinant of nosocomial infection? Current Opinion in Infections Diseases, 17, 4, 329. Karsh, B-T. and Brown, R. in press, Macroergonomics and patient safety: the impact of levels on theory, measurement, analysis and intervention in medical error research. Applied Ergonomics. Parker, S.K. 2007, Job satisfaction. In S. Rogelberg, (Ed.) The Encyclopaedia of Industrial and Organizational Psychology. (New York: Sage). Perrow, C. 1999, Normal Accidents. (Princeton: Princeton University Press). Rasmussen, J. 1997, Risk management in a dynamic society: a modelling problem. Safety Science, 27, 183–213. Reason, J. 1995, A systems approach to organizational error. Ergonomics, 38, 1708–1721. Turner, B.A. 1978, Man-Made Disasters. (London: Wykeham Publications). Vaughan, D. 1996, The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA. (Chicago: Chicago University Press). Vicente, K.J. and Christoffersen, K. 2006, The Walkerton E. Coli outbreak: a test of Rasmussen’s framework for risk management in a dynamic society. Theoretical Issues in Ergonomics Science, 7, 2, 93–112. Waterson, P.E. 2008, Taking stock of the systems approach to patient safety. In S. Hignett, B. Norris, K. Catchpole, A. Hutchinson and S. Tapley (Eds.), Improving Patient Safety. (Loughborough: Ergonomics Society). Weick, K.E. and Sutcliffe, K.M. 2003, Hospitals as cultures of re-enactment: a re-analysis of the Bristol Royal Infirmary. California Management Review, 45, 2, 73–84. West, M.A., Borrill, C., Dawson, J., Scully, J., Carter, M., Anelay, S., Patterson, M. and Waring, J. 2002, The link between the management of employees and patient mortality in acute hospitals. International Journal of Human Resource Management, 13, 8, 1299–1310. Wood, S. and Wall, T.D. 2002, Human resource management and business performance. In P.B. Warr (Ed.), Psychology at Work (5th edition). (Harmondsworth: Penguin Books). Zohar, D. 2000, A group-level model of safety climate: Testing the effect of group climate on microaccidents in manufacturing jobs. Journal ofApplied Psychology, 85, 587–96.
PORTABLE PODS: DESIGN FOR UNSCHEDULED (URGENT) CARE IN THE COMMUNITY S. Hignett1 , A. Jones1 & J. Benger2 1
Healthcare Ergonomics and Patient Safety Unit, Department of Human Sciences, Loughborough University, UK 2 Academic Department of Emergency Care, University of the West of England, Bristol, UK In the last five years UK government policy has moved towards increasing the provision of unscheduled (urgent) care in the community. To meet these changes the emergency ambulance service is shifting from an organization designed to convey patients to hospital to a professional group capable of assessing urgency and delivering the appropriate treatment to the patient; ‘providing the right response, first time, in time’. This paper explores modular design for supporting (portable) technology and draws on examples from emergency care practitioner services (Ambulance Trusts), out-of-hours GP services and Minor Injuries Units (Primary CareTrusts), and Emergency Departments (Acute Trusts) to investigate the workplace layout and clinical activities, including the use of equipment and consumables.
Introduction In May 2004 the Department of Health (DH) commissioned a strategic review of National Health Services (NHS) ambulance services in England, focusing on how the ambulance service could shift from providing resuscitation, trauma and acute care towards “Taking healthcare to the patient: transforming ambulance services in the community” (DH, 2005a). The strategic review set out a five-year vision to: • Improve the speed and quality of call handling, ‘hear and treat’. • Increase the range of mobile healthcare services for patients requiring urgent care, ‘see and treat’. • Increase the range of other services such as diagnostics and health promotion. • Improve the quality and speed of service to patients with emergency care needs. In 2007/08 there were 7.2 million emergency and urgent ambulance calls in England, with 81% (5.9 million) resulting in an emergency response and 4.26 million patient journeys from 12 ambulance Trusts (NHS Information Centre, 2008). Before 1st April 2007 emergency and urgent calls were prioritized and classified into four categories: • A, immediately life threatening (8 minute response); • B, serious but not life threatening (19 minute response);
174
Portable pods
175
• C, not immediately serious or life threatening (locally agreed response time standards); • Urgent, usually in response to a GP/midwife/health care professional request for transportation via a separate phone line (NHS Information Centre, 2008). After 1st April 2007 the urgent calls have been included in the emergency call system to ensure that all patients are triaged within the same response time target framework (DH, 2005a). Urgent calls are usually classified as a category C, accounting for 40.4% of all calls in 2007/08. The response standard was further changed from 1 April 2008 (‘Call Connect’), with the clock start changed to the time of the call connection to the ambulance control room rather than when key information had been obtained from the 999 caller, with the aim of increasing survival rates for cardiac arrest patients (DH Ambulance Policy, 2008). Urgent Care is defined as ‘the range of responses that health and care services provide to people who require – or who perceive the need for – urgent advice, care, treatment or diagnosis. People using services and carers should expect 24/7 consistent and vigorous assessment of the urgency of their care need and an appropriate and prompt response to that need’ (DH, 2005b). In 2007 the Royal College of General Practitioners commented that the organization of urgent and unscheduled care services needed to improve to address the fragmentation of care services including duplication of provision, wasted resources and unnecessary handoffs between providers. They suggested the adoption of a ‘whole systems approach’ to ensure integration of services and the creation of virtual urgent care centres for different providers to work together (Lakhani et al., 2007). Emergency and Urgent Care Networks (EUCNs) were set up to co-ordinate and organize a diverse group of primary, secondary, out of hours (ambulance) and social services. The care board structure included a range of stakeholders from acute, primary care, ambulance, NHS Direct, mental health (crisis teams), social care, and patient representation. Some EUCNs also included police, passenger transport executive, local authority, public health, health promotion, community pharmacy (Turner et al., 2008). Emergency and urgent care services are faced with the triple hurdle of delivering a service that is more responsive, more resource efficient, and that also employs the latest medical technologies (Phillips and Caldwell, 2007). In 1999 the role of Emergency Care Practitioner (ECP) was developed in the Ambulance Service. This role aimed to raise the clinical skills of paramedics, ensuring that patients receive the right care at the right time, and in the right place (DH, 2004). Snooks et al., (2004) reviewed the early operational activities of ECPs and found no significant difference in the rate of conveyance, but that a longer time was spent with patients, generating a more in-depth assessment with more comprehensive clinical records. However by 2007 a different picture was emerging, with Mason et al. (2007) finding that ‘overall ECPs carried out fewer investigations, provided more treatments and were more likely to discharge patients home than usual providers’, suggesting that the increased clinical skills were achieving the goal of delivering the appropriate treatment to the patient. In order to achieve the vision of a modernized workforce that is able to provide a greater range of mobile urgent care, ambulance clinicians will need a wider range
176
S. Hignett, A. Jones & J. Benger
of competencies and underpinning knowledge whilst maintaining the vocational nature of their training (DH Ambulance Policy, 2008). Many Trusts are changing the profile of their workforce in line with the recommendations including consideration of the interface between people, processes and technology using a sociotechnical framework (DH, 2005a). However an initial review of the changed response/service model has identified potential risks including poor state of fleet and equipment and poor systems for integrating community responders (DH Ambulance Policy, 2008).
Project aim The vision for this project was to support the effective delivery of urgent care in the community by developing a system of configurable, rapidly exchangeable pods ready-loaded with equipment and consumables (portable pods) to suit various types of presenting complaints; attached to base vehicles (mobile pods) for delivery to the point of care at a variety of locations. It was proposed that pods could be exchanged, or remain with the vehicle; used as multiples for major incidents with a view to broadening their function beyond the current emergency mode of use. Figure 1 shows the conceptualization of the project, with a central focus on clinical activities (in the ergonomics work package) to: 1. Understand and identify current and future care activities in emergency departments, minor injury units, ambulances and out-of-hours GPs which could be delivered in the community. 2. Define the configurations for the portable and mobile pods. A previous paper (Jones et al., 2008a) described the process for stakeholder participation to identify and categorise clinical complaints for treatment and care in the home/community by ECPs as physical minor, physical uncertain, physical major, social, mental and elective. A set of 6 presenting complaints were selected for further study (JB): breathing difficulties (physical minor); chest pain (physical uncertain); lacerations (physical minor); falls (physical uncertain/social); neck pain
Figure 1.
Project overview.
Portable pods
Table 1.
177
Participants at 2007 workshops (n = 22).
Organisation
Participants (job title)
Acute
Emergency Department (ED) Registrar; ED Charge Nurse; Emergency Nurse Practitioner (2); Research Nurse GP; PCT Chair; Out-of = Hours Nurse Practitioner; MIU Nurse Practitioner (2); Medical Director (Deputy CEO); Strategic Lead Occupational Health Services; Nurse Advisors (2); Change Manager Assistant Director of Corporate Services; Fleet Manager; Service Improvement Manager; Back Care Advisor; Emergency Care Practitioner (3)
Primary Care
Ambulance
(physical minor); head injury (physical minor) (Jones et al., 2008b). This paper will describe the iterative process used to develop the design specification for portable pod through stakeholder workshops, responder bags audits, observations and design decision groups.
Method Data were collected using stakeholder workshops, audits and clinical observations. In 2007 two workshops were held with 22 stakeholders participating from acute, community and ambulance NHS Trusts (East Midlands and South West). They identified categories of complaints and lists of equipment and consumables for the portable treatment pods. In 2008 a further workshop was held with fleet (4), clinical (5), service (4) and health and safety managers (2) from 5 Ambulance Trusts to present the findings of the 2007 workshops, audits and observations. Data were collected as a series of semi-structured questions in individual workbooks, for example: • How might medical equipment used by the ambulance service be: 1. Better organised, laid out and accessed in vehicles? 2. Effectively rationalised to provide what is needed without carrying excessive or unnecessary stock? • How do you see the role of portable kit and diagnostics changing: 1. By 2015 2. By 2025? The written responses were entered into NVivo (Bazeley and Richards, 2000) for analysis. NVivo is a qualitative data management programme that support coding, searching and theorizing. An audit of portable equipment (used by Fast Responders/Paramedics) was carried out in 2002 (Redden, 2002). This was updated in 2008 for ECPs (Reynolds, 2008). Observational data were collected from 84 patients at EDs and Minor Injuries Units presenting with the 6 complaints identified from the 2007 workshops (Jones et al., 2008a). Data were recorded with Link Analysis (Jones et al., 2008b) with supplementary field notes for equipment and consumables requirements.
178
Table 2.
S. Hignett, A. Jones & J. Benger
Observational data for equipment and consumables from EDs and MIUs.
Treatment groups
Equipment/Consumables
Minor wounds
Dressing pack, Irrigation fluid, forceps, scissors, tissue glue, steristrips, suture kit (sutures and instruments), Ear, nose and throat Tongue depressors, thermometer (tympanic), suction, auroscope Respiratory Oxygen, masks (including tracheotomy), stethoscope, pulse oxymeter, nebuliser, nasal and oral airways, peak flow meter, suction kit Blood monitoring Specimen bottles, phlebotomy kit, blood pressure cuff, cannula (various), giving sets, blood glucose testing strips, Eyes Irrigation fluid, eye pads Basic life support Defibrillator, oxygen, face mask, electrocardiogram (ECG) capability (computerized transmissible) Communications: Mobile telephone, tele-medicine capability, on-line decision support software Urinary Urinalysis kit, sample bottles, catheter equipment, incontinence pads General Gloves (sterile/non sterile), neck collar, bandages, gauzes, dressings, waste bins (sharps, clinical, domestic), sphygmomanometer, skin preparation wipes, apron, handwash facilities, tissues, syringes, needles, lubricant, magnifying glass, razor, tweezers, scissors, referral letter/x-ray/prescription (prescribing guidelines), patella hammer, ring cutter, safety glasses, helmet Drugs Glyceryl trinitrate (GTN) spray, aspirin, intravenous fluids, paracetamol, anti-inflammatories, antibiotics, local anaesthetics
Table 3.
Problems and suggested design solutions for the portable pod (workshop 2008).
Problems/Barriers Individual preferences resulting in duplication, different equipment (and quantities) and different locations Weight and size of bags for carrying (stairs, long distances) Infection control risks
Design ideas/solutions Standardisation of contents and layout Organise into clinical ‘jobs’ (cardiac, breathing difficulties) Miniaturise and modularise diagnostic equipment (reconnect to base station in vehicle, ‘wi fi’) Smaller bags (‘snatch/grab’ bags) Disposable bags for individual patients Equipment in safe, sterile, disposable containers
Results The 2007 workshops produced a list of equipment and consumables, grouped by clinical criteria. These were compared with the audits and the observational data to give a proposed generic list of equipment and consumables (table 2). The workshop in 2008 reviewed the findings. The analysis of the workbooks resulted in a number of thematic codes including problems/barriers and design ideas/solutions for the portable pod (table 3).
Portable pods
Figure 2.
179
Design idea for the portable pod.
These findings will be reviewed at Design Decision Groups (DDGs) to be held in November/December 2008 with operational staff (ECPs and Fast Response Paramedics) and management/staff representatives (Equipment Working Group) from EMAS. The DDG format will both challenge current practice (with word maps and round robin questionnaires) and support innovative re-design through mock-ups/prototypes (Wilson et al., 2005). Staff will be given an opportunity to prepare bags for specific care pathways. These will be reviewed and discussed in small group sessions. The first session will finish with a drawing exercise to create portable pods with improved functionality and usability. For the second session, prototypes (example in figure 2) based on the outputs from session one will be manufactured and used as the focus for the discussion and modification of the design specification.
Discussion The concept of modularisation has been called the goal of good design (Gershenson et al., 1999), with benefits including ease of product updating, increased product variety, and ease of design and testing. Modularisation has also been described as a ‘general systems concept’ (Schilling, 2000), with modular products as ‘systems of components that are ‘loosely coupled’. The challenge for the design of modular products/systems is to maximise flexibility, with staff being willing and able to distinguish between the performance, quality and value attributes of different components (e.g. equipment and consumables). Poland et al., (2005) questioned whether interventions (e.g. standardized equipment/consumables) would necessarily bring benefits. They suggested that professional work is ‘constructed as requiring the identification of best practices through careful and rigorous empirical evaluative research and ‘applying’ these as faithfully as possible’, however it is important to retain clinical autonomy in diagnosis and treatment so a degree of flexibility will be necessary to support individual variation (both clinical practitioner and patient). A modular design, including ‘grab bags’ may provide the optimal solution; this design approach will be further explored. A greater challenge will be the dissemination of the research findings. The intricacies of the UK healthcare market mean there is no central point of access for innovations and there is no central unit responsible for ensuring innovations are
180
S. Hignett, A. Jones & J. Benger
considered or taken up (Phillips and Caldwell, 2007). A study commissioned by the Association of British Healthcare Industries (ABHI, 2002) found the behaviour of the NHS to be the biggest limiter to growth, including a limited capacity to innovate as a customer due to, for example, a lack of mechanisms for trialing/evaluating clinical and cost effectiveness; a lack of staff motivation to innovate due to day-today pressures and a blame culture that penalises innovation; a lack of dissemination of good practice between Trusts. We are involved with the National Ambulance Strategic Procurement Team so hope that this, together with professional and research dissemination will lead to the successful implementation of the portable pod.
Acknowledgements We would like to thank the EPSRC for funding the Smart Pods project and our partners at the Royal College of Art, Bath and Plymouth Universities. We would particularly like to acknowledge the contribution of the 6 participating NHS Trusts: University Hospitals Bristol NHS Foundation Trust (UHBT), University Hospitals Leicester Foundation NHS Trust (UHL), BrisDoc, Leicester County and Rutland Primary Care NHS Trust (LCRPCT), Great Western Ambulance Service NHS Trust (GWAS) and East Midlands Ambulance Service NHS Trust (EMAS).
References ABHI, 2002. A competitive analysis of the healthcare industry in the UK; CoMap II. An independent report commissioned by the Association of British Healthcare Industries. Bazeley, P., Richards, L. 2000, The NVivo®Qualitative Project Book, (Sage Publications, London) DH, 2004, The Emergency Care Practitioner report – Right skill, Right time, Right place, http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/ PublicationsPolicyAndGuidance/DH_4093086 (Accessed 1 April 2008) DH, 2005a, Taking Healthcare to the Patient. Transforming NHS Ambulance Services. http://www.dh.gov.uk/assetRoot/04/11/42/70/04114270.pdf (Accessed 2 Sept. 2005) DH, 2005b, Urgent Care: Direction of Travel. Consultation Document, (Department of Health, London) DH, 2008, Guidance on Pre-registration Education and Funding for Paramedics, (Department of Health, London) DH Ambulance Policy, 2008, Changing Times: sustaining long-term performance against ‘Call connect’ for NHS Ambulance Services, (Department of Health, London) Gershenson, J.K., Prasad, G.J., Allamneni, S. 1999, Modular Product Design: A Life-cycle View Journal of Integrated Design and Process Science, 3, 4. http://www.me.mtu.edu/∼jkgershe/lel/research/JIDPS.pdf (Accessed 26th August 2008)
Portable pods
181
Jones, A., Hignett, S., Benger, J. 2008a, Identifying current and future care activities in ambulances, emergency departments and primary care. Emergency Medicine Journal, 25 (Suppl):A3 Jones, A., Hignett, S., Benger, J. 2008b, A Comparison of Urgent Tasks and Workspace Design in Pre-hospital Care. In Hignett, S., Norris, B., Catchpole, K., Hutchinson, A., Tapley, S. (Eds.) 2008, From Safe Design to Safe Practice. Proceedings of the Improving Patient Safety conference. The Ergonomics Society. 16–18 July, Cambridge. 121–125 Lakhani, M., Fernandes, A., Archard, G. 2007 Urgent Care: a position statement from the Royal College of General Practitioners, ( Royal College of General Practitioners, London). Lattimer, V., Brailsford, S., Turnbull, J., Tarnaras, P., Smith, H., George, S., Gerard, K., Maslin-Prothero, S. 2004, Reviewing emergency care systems 1: insights from system dynamics modelling. Emergency Medicine Journal 21, 6, 685–691. Mason, S., O’Keeffe, C., Coleman, P., Edlin, R., Nicholl, J. 2007, Effectiveness of emergency care practitioners working within existing emergency service models of care. Emergency Medicine Journal 24, 239–243 NHS Information Centre, 2008, Ambulance Services, England, 2007–08, (The Information Centre, London) Phillips, W. and Caldwell, N. 2007, Supplying Innovative Healthcare Technologies into the UK Healthcare Sector: A literature review. Internal document. Centre for Research in Strategic Purchasing and Supply, University of Bath School of Management Poland, B., Lehoux, P., Holmes, D., Andrews, G. 2005, How place matters: unpacking technology and power in health and social care, Health and Social Care in the Community, 13, 2, 170–180 Redden, D. 2002, An Evaluation of Fast Response Vehicle Paramedic Bag Systems. Unpublished M.Sc. Dissertation. Dept of Human Sciences, Loughborough University Reynolds, R. 2008, Standardisation of Emergency Care Practitioners Equipment and Consumables. Unpublished B.Sc. Dissertation, Dept of Human Sciences, Loughborough University Schilling, M.A. 2000, Toward a general modular systems theory and its application to interfirm product modularity, Academy of Management Review, 25, 2, 314–334 Snooks, H., Kearsley, N., Dale, J., Halter, M., Redhead, J., Cheung, W.Y. 2004, Towards primary care for non-serious callers to the emergency ambulance service: results of ‘Treat and Refer’ protocols for ambulance crews, Quality and Safety in Healthcare, 13, 435–443 Turner, J., Nicholl, J., O’Caithain, A. 2008, A preliminary study of Emergency and Urgent Care Networks. http://www.dh.gov.uk/en/Consultations/ Responsestoconsultations/DH_080369 (Accessed 25th August 2008) Wilson, J.W., Haines, H., Morris, W. 2005, Participatory Ergonomics, in Wilson J.W. and Corlett, E.N. (Eds.) Evaluation of Human Work, Third Edition, (Taylor and Francis, London) 933–962
USING USABILITY TESTING AS A BENCHMARKING TOOL – A CASE STUDY ON MEDICAL VENTILATORS Y. Liu & A. Osvalder Division Design, Department of Product & Production Development, Chalmers University of Technology, Gothenburg, Sweden Medical ergonomics has become an important topic in the field of ergonomics due to the increasing deficiencies in medical device design. The aim of the present study was to use usability testing as a benchmarking tool of modern and complex ventilator machines in a real hospital setting to propose future redesign ideas for a target machine – SERVO-i. A usability study was carried out at the neonatal intensive care unit (ICU) with 6 expert nurses. During the tests, an artificial lung was used to simulate the real treatment context on a neonatal patient. A number of potential usability problems were detected on SERVO-i. Using Babylog and Stephanie as two reference machines, the strengths and weaknesses of SERVO-i were identified as well. Some valuable lessons were learnt from the study, which could be used as guidance for the choice of test subjects and control of tests in real life human-machine working environments.
Introduction In recent years, various medical accidents have been reported. However, the number of accidents, incidents and near mistakes with medical equipment is much larger than the number reported officially (Gaba et al., 1995). Reports of critical mishaps show that misuse and human error are more frequently involved than equipment failures (Hyman and Schlain, 1986; Craig and Wilson, 1981; Cooper et al., 1978). Concerning human errors in hospitals, Russell (1992) indicated that avoidable mistakes killed 100,000 patients a year. Human error in medicine, and the adverse events that might follow, are related to problems of psychology and engineering, rather than those of medicine (Senders, 1994). Medical ergonomics has become an important topic in the field of ergonomics. Medical devices play important and life-sustaining roles in the health care sector, especially in critical-care medical settings, such as Intensive Care Units (ICUs), Anesthesia Workstations, and Operating Rooms (ORs). Deficiencies in medical device design were increasingly identified and addressed in recent years. As is widely accepted, design of medical devices concerns not only the quality of the medical care delivery, but also the patients’ safety and health. Nowadays, usability and risk management in the medical field become an important topic of human factors research. Manufacturers have realized that safe and effective use of a medical device depend on the interaction between the operating environment, user 182
Using usability testing as a benchmarking tool
183
capabilities, stress levels, and device design, therefore usability issues have to be considered in the design process (Sawyer, 1996). Due to the academic research in usability engineering, the focus of risk analysis and risk management has already been shifted from technical aspects and human performance to human-machine interaction (HMI). How to improve safety and achieve user-friendly goals by detecting design deficiencies and reducing errors in the design process has increasingly drawn attention of designers and manufacturers. If design deficiencies or human errors could be detected during evaluations in the design process, numerous medical accidents or injuries could be avoided and more lives, money and labor could be saved. Ventilator machines are used as essential and critical medical devices in Anesthesia Workstations, ICUs, and ORs. How to improve the usability of ventilator machines attracts designers’ and ventilator manufacturers’ attention today, and thus how usability tests should be performed. SERVO-i has been an advanced ventilator machine used normally for adult patients. However, the manufacturer of SERVO-i chose it as a target machine for further development due to their future marketing strategy for neonate patients in neonatal ICUs at hospitals. The purpose of the present study was: (1) to detect potential usability problems of a modern complex ventilator machine (SERVO-i) for use in a neonatal ICU, and (2) to investigate the strengths and weaknesses of the target machine (SERVO-i) compared to two reference ventilator machines (Babylog & Stephanie). The aim of the study was to provide an example of using usability testing as a benchmarking tool of ventilator machines in a real hospital setting to propose future redesign ideas.
Methods and materials The usability study was carried out at the neonatal ICU in Queen Silvia Children’s Hospital in Gothenburg, Sweden. In the neonatal ICU, ventilator machines are frequently used on premature and neonate babies (0–3 months old), who require critical medical treatment. 6 expert nurses from the neonatal ICU, who had worked with 2 to 6 different brands of ventilator machines for 10 years at average, participated in the usability tests. All test participants were expert users on Babylog and Stephanie, but had never used SERVO-i before. The three ventilators had different types of interfaces. SERVO-i had a highly sensitive touch-screen interface, Babylog had a monochrome screen and a few hard buttons for menu choice, and Stephanie had a colorful screen and a large scroll knob for software menus (Figure 1). 6 significant task scenarios were selected with the help of a pediatrician specialist. The task scenarios were classified into two categories according to frequency of use, i.e. (I) Frequently used tasks and (II) Infrequently used tasks. Category (I) included Task 1 – Starting ventilation, Task 2 – Alarm handling, and Task 3 – Checking critical parameters. Category (II) included Task 4 – Trigger handling, Task 5 –Displaying ventilation process in diagrams, and Task 6 –Displaying alarm history.
184
Y. Liu & A. Osvalder
Figure 1. The three ventilators included in the usability study.
The usability tests were conducted according to a general test procedure proposed by, for instance, Nielsen (1993) and McClelland (1995). During the tests, an artificial lung was used to simulate the real treatment context on a neonatal patient. Prior to the usability tests, two pilot tests were carried out to verify the test procedure. At the beginning of the test session, each test subject was informed about the purpose of the test and given short instructions regarding the test procedure. Two test sessions were conducted with each test subject, one session on the reference machines (i.e. Babylog & Stephanie), the other session on the target machine (SERVO-i). Each test session took approximately one hour. During the tests, video cameras were used to record the whole test procedure; however, all the test subjects were notified that only their hands would be recorded, thus ensuring anonymity. During the tests, the number of errors was collected from each task scenario. At the end of each test session, an after-session interview was held with the test subjects to get their comments and suggestions on the interface design of the ventilators. A simple 10-item scale questionnaire, the System Usability Scale (SUS) questionnaire (Brooke, 1996), was also used to get each test subject’s overall satisfaction of each ventilator. At the end of the second test sessions, a psychometric technique called paired comparisons (Thurstone, 1927) was used in order to get the test subjects’ subjective assessment on the interface design of three ventilators. Pictures of the three ventilator interfaces were presented in pairs to each test subject. The test subjects had to select which interface they thought better, until all possible paired combinations had been used.
Results The results were based on the data analysis of the test subjects’ behavior and their verbal comments. A number of potential usability problems were detected on SERVO-i, such as (1) Sensitivity problem of the touch screen;
Using usability testing as a benchmarking tool
185
Number of failures
7 6 5 4
Babylog
3
Stephanie
2 1
Figure 2.
6
5
sk Ta
4
sk Ta
3
sk Ta
2
sk Ta
sk Ta
Ta
sk
1
0
Number of failures on Babylog and Stephanie.
(2) Inconsistency confirmation methods; (3) Difficult to get an overview of all functions; (4) Bad symbol of ventilation start button; (5) Bad information organization on the screen; (6) Bad readability of measured values; (7) Lack of ‘Help’ function; and (8) Terminology problem. Then number of failures during completing the frequently used tasks (i.e. Task 1, Task 2, and Task 3) and the infrequently used tasks (i.e. Task 4, Task 5, and Task 6), were collected and analyzed both for Babylog and Stephanie (Figure 2). In general, more errors were shown in the infrequent-used tasks for both machines. Although the test subjects were novice users on SERVO-i, the results of the paired comparison method implied that SERVO-i was most preferable due to better information clarity, information layout, adequate amount of relevant information, and ease of understanding. However, Babylog was most preferable due to its ease of use. Concerning overall satisfaction, a within-subjects’ Friedman test was conducted. No statistical significant difference was shown between SERVO-i and the two reference machines (p = 0.74). Nevertheless, higher ranking was found on SERVO-i by the test subjects who were first time users, which implied the test subjects’ positive attitude to SERVO-i. According to the after-session discussion, the test subjects commented freely on the pros and cons of the three ventilator machines, which provided useful information for future redesign. The strengths of SERVO-i included modern touch screen design, clear information display and value settings, ease of navigation, and logical menu structure. Sensitivity problem of the touch screen, bad navigation path to ‘Mode’ settings, bad readability and clarity of the ‘Trend’ display, were counted as weaknesses of SERVO-i. Although the strengths and weaknesses of the two reference machines were not the focus of this study, the results could be used to get an overall picture about competitors’ products and their competencies. For instance, the strengths of Babylog were counted as ease of use, simplicity of interface, and adequate amount of information on the display, while the strengths of Stephanie were ease of navigation, clear display, ease of use in parameter settings, and a good overall picture of all
186
Y. Liu & A. Osvalder
information on one screen. The information could be used as useful implications for future redesign and marketing strategy.
Discussion In the present study, 6 expert nurses who had in average 10 years working experience with 2–6 different types of ventilator machines, interacted with a new ventilator machine (SERVO-i) as first time users. The results showed that the test subjects were very skilful in frequently used tasks and were almost frustrated on infrequently used tasks, when interacting with the familiarized machines that they were used to in their daily work. This phenomenon conforms to Nielsen’s (1993) opinion that even an expert user might be quite novice with respect to infrequently used tasks. In other words, expertise of an expert user might be task-bounded. The results posed a practical question concerning how to define expert users in the medical field. In the present study, the test subjects were actually task-experts rather than machine-experts. In the medical health care sector, time and expertise concern safety and health during the treatment. In case of emergency treatment, it is impossible for health carers or nurses to get time and look up the users’ menu about some infrequent-used functions of the medical device. Thereby, it is important that expert users of medical devices should be skilful in all available functions rather than partial functions of the devices. However, the after-session discussion revealed a special working organization factor at the neonatal ICU, that is, the medical doctors dominated in the work with ventilation treatment, while ventilator nurses were only subjected to the doctors’ permission during the treatment process. To some extent, such convention determines or affects constituent of nurses’ expertise. However, whether such affects should be counted as positive or negative is a question to the medical health care sector. During the usability evaluation, sensitivity problem of the touch screen was found as an important and typical problem on SERVO-i, which affected the test subjects’ performance in some ways, such as distinguishing and judging the interactive items and non-interactive items on the screen. In addition, the test subjects might have an inappropriate mental model concerning how to interact with some graphical items on the screen due to the sensitivity problem. Many of the test subjects thought that they should use the large adjustable knob rather than the touch screen to activate graphical items. This implied that the users’ mental models and performance were basically influenced by the features of the touch screen. Another typical usability problem detected was related to terminology. The results revealed that terminology differences between the ventilators confused the test subjects. Due to their previous accumulated experience and knowledge, expert users are always assumed to have better guessability (Jordan, 1998) to cope with the differences in the components and layout of a new or unfamiliar interface. However, the terminology difference hinders the effect of guessability. Consequently, the expert users might be confused and thereby hesitate to make a trial on further interactions. This posed a question to the anaesthesia and ventilation field: Is it possible to have common or standard terminologies on all ventilator products for
Using usability testing as a benchmarking tool
187
designers and manufacturers? Without doubt, standardization on terminology can reduce unnecessary confusion or errors in navigation. Since the test subjects used Babylog and Stephanie in their daily work, they were familiar with and used to work with these machines. SERVO-i was new and unknown to the test subjects. It is easy to hypothesize that the test subjects might be influenced by their experience or habits on the familiarized machines. However, the results of the subjective assessments did not prove this hypothesis. Instead, the results implied that expert users are not always biased or subjected to the constraints of their previous experience in subjective assessments. Some valuable lessons were learnt from the study, which could be used as guidance for choice of test subjects and control of tests in real life human-machine working environments. In the present study, the best way to perform usability tests was to make a test schedule in the morning, which depended on the available staff and how much activity there currently was at the neonatal ICU. This had to make because of the typical feature of ICUs at hospitals, i.e. an intensive and stressful working environment. Due to its strength in evaluating handling characteristics, usability test is suggested to be performed after the clinical trial during purchase stage (Liljegren and Osvalder, 2004). To perform usability studies at hospitals requires a lot of flexibility in time schedule and planning. It is hard to arrange a reliable schedule since everything is very much event driven in this type of working environment. Therefore, adequate negotiation with both high-level managers and nurses is necessary and important for performing a successful usability study in a hospital setting. In addition, it was found difficult to get a large number of test subjects in ICUs due to the stressful and event driven working environment. However, Nielsen (1993) indicated that 5 test subjects are enough in empirical usability evaluations in terms of reliability or accuracy level. With 5 test subjects, a 70% confidence probability of getting within ±15% of the true mean could be gained (Nielsen, 1993).
Conclusion The intention of the present study was to use usability testing as a benchmarking tool to evaluate the current design of SERVO-i in a real hospital setting, with the focus on investigating first time use experience by expert ventilator nurses. Based on the results, the following conclusions can be drawn: 1. The potential usability problems of SERVO-i for future improvement included: • sensitivity of touch screen • consistency of confirmation methods • overview of all functions • symbol of ‘Start’ button • information organization • readability of measured values • ‘Help’ function • visualization of interaction • terminology
188
Y. Liu & A. Osvalder
2. SERVO-i was preferable to the two reference machines in terms of better information clarity, information layout, adequate amount of relevant information, and ease of understanding. 3. Referring to the two reference machines, the ‘ease of use’ aspect should be improved on SERVO-i. 4. The terminology difference between the same type of medical devices from different manufacturers might be an important factor affecting expert users’ performance in the interaction. 5. Expert users might not always be biased or subjected to the constraints of their previous use experience when giving subjective assessments on a new or unknown medical interface. 6. The present study provided a tentative example of how to carry out benchmark tests concerning usability aspects on medical equipments in a hospital setting.
References Brooke, J. 1996, SUS – A quick and dirty usability scale. In P.W. Jordan et al. (eds.) Usability Evaluation in Industry, (Taylor & Francis, London) Cooper, J., Newbower, R.S., Long, C.D. and McPeek, B. 1978, Preventable anesthesia mishaps: a study of human factors, Anesthesiology, 49, 399–406 Craig, J. and Wilson, M.E. 1981, A survey of anesthetic misadventures, Anesthesia, 36, 933–936 Gaba, D.M., Howard, S. and Small, S. 1995, SituationAwareness inAnesthesiology, Human factors, 37(1), 20–31 Hyman, W.A. and Schlain, L.A. 1986, User problems and medical device recalls, Medical Instrument, 20, 14–16 Jordan, P.W. 1998, An Introduction to Usability, (Taylor and Francis, London) Liljegren, E. and Osvalder, A.L. 2004, Cognitive engineering methods as usability evaluation tools for medical equipment. International Journal of Industrial Ergonomics 34(1), 49–62 McClelland, I. 1995, Product assessment and user trials. In J.R. Wilson and E.N. Corlett, (eds.) Evaluation of Human Work, (Taylor and Francis, London) Nielsen, J. 1993, Usability engineering. (Academic Press, San Diego) Russell, C. 1992, Human error: Avoidable mistakes kill 100,000 patients a year, Washington Post, Feb. 18, WH7 Sawyer, D. 1996, Do it by design: An introduction to human factors in medical devices. U.S. Department of Health and Human Services, Food and Drug Administration (Centre for Devices and Radiological Health) Senders, J.W. 1994, Medical devices, medical errors, and medical accidents. In M.S. Bogner (ed.) Human error in medicine, (Lawrence Erlbaum Associates Inc., Publishers, New Jersey), 159–177 Thurstone, L.L. 1927, A law of comparative judgment. Psychological Review, 34, 278–286
DOES THE OLDER WORKFORCE WITH HIGH WORK DEMANDS NEED MORE RECOVERY FROM WORK J.J. Devereux1 & L.W. Rydstedt2 1
University of Surrey, UK University West, Sweden
2
A follow-up study investigated how psychological and physical work demands affected older workers need for recovery at the end of a working day. A comparison of need for recovery results measured at time 1 and time 2, fifteen months apart, showed similar results. Psychologically demanding work was the stronger risk factor compared to high physical work demands. Older workers were more likely to have a significantly higher need for recovery from work after performing psychologically and physically demanding work compared to younger workers.
Introduction Approximately one third of the UK labour force will be age 50 or over by the year 2020. According to the Office of National Statistics, ageing of the workforce will be the most significant development in the labour market over the next 15 years. As ageing in the population increases and the need to work longer into later life takes effect in the workplace, it will become more important to understand the needs and physical and mental abilities of workers performing different kinds of work. Older workers might be at risk of overload from the physical and psychological demands of work. Besides working into older age, there is also the added complication of having the workforce adapt to different organisational work patterns including flexible working hours, the 24/7 work culture and irregular shifts. These changes in the workforce and the organisation of work may have serious implications for an increased risk of cardiovascular disease, subjective health complaints, musculoskeletal disorders and duration of sickness absence. Recent studies have used the concept of need for recovery from work to explore the temporary overload from the physical and psychological demands of work. It is viewed as an individual perception of the time required to recover from adverse working conditions that results in insufficient unwinding after exposure to stressful work characteristics and is a strong predictor of subsequent cardiovascular disease (van Amelsvoort et al., 2003), perceived job stress (Devereux & Rydstedt, 2008), subjective health complaints and duration of future sickness absence (Sluiter et al., 2003) In the study by Sluiter et al., (2003) the authors recommended that age be treated as an effect modifier between work demands and need for recovery after work. 189
190
J.J. Devereux & L.W. Rydstedt
The purpose of this study, using longitudinal data, was twofold. First, investigate the relationship between physical and psychological work demands and subjective need for recovery after work reported at two different time points 15 months apart. And secondly, determine whether age modified the risk relationship between the physical and psychological demands of work and subjective need for recovery after work. Ethical approval was given for this study by the University of Surrey Committee on Ethics.
Study population The respondents were all participating in a large longitudinal study on work stress and work-related musculoskeletal and mental disorders (Devereux et al., 2004). This comprised a baseline cross-sectional study of 8,000 workers of whom 3,139 were followed for 15 months (approx.). The cohort was drawn from 20 organizations across 11 industrial sectors in the U.K. The occupational groupings from the study were compared to the Standard Occupational Classification 2000 system using the 9 major groups. The study sample had an over-representation of professional workers and an under-representation of skilled trade, personal service, sales customer service and elementary workers. This was partly caused by the different response rates across different occupations. There was no age or gender difference between the study population of 3139 and the total population of 8000. Male and female workers within the age range 17–69 years were included in the study. Part-time workers and workers principally based long term within client organisations were excluded from the study sample. Exposure data, demographics and need for recovery were assessed using a questionnaire at baseline (time 1), 3056 workers had valid need for recovery and age scores. After attrition mainly due to turnover and downsizing in some of the organisations, the sample for the resurvey at time 2 consisted of 2091persons with valid need for recovery scores, about 68% of the participants at the baseline survey (time 1). The attrition did not affect the age and gender distribution of the sample.
Measures The ‘need for recovery’ scale consists of 11 dichotomous items concerning recuperation after one day of work. Examples of items are ’my job causes me to feel rather exhausted at the end of the working day’ and ’after a working day I am often too tired to start and activities’, and ’sometimes, I cannot optimally perform my job because of fatigue during the last part of the working day’. This scale has been evaluated against neuroendocrine activity and subjective health complaints (Sluiter et al., 2001). Furthermore, the test-retest reliability over a 2 year period in a stable work environment is good to excellent (Intraclass correlations 0.68–0.80) (de Croon et al., 2006). All items were coded so that higher scores meant more complaints, i.e. more need for recovery after work. Participants with missing data on any of the 11 items were excluded from analysis. The Cronbach’s alpha of the entire scale was 0.85. Median split was used to contrast employees with low and high levels of need
Does older workforce with high work demands need more recovery
191
for recovery after work and work content variables. All other measures have been reported previously (Devereux et al., 2004). Job demands (Chronbach alpha 0.66) was assessed using the modified Job Content Questionnaire comprising 4 items (working very fast, intensively, having enough time and difficulty combining tasks). Extrinsic effort was assessed using the Effort-Reward questionnaire. Extrinsic effort items were rated on a dichotomous (agree/disagree) scale. Extrinsic effort was measured by 6 items including the item concerning physical loading. Extrinsic effort without the physical demands item was also used, as a large proportion of the study population performed sedentary work. The physical demands item was also reported separately. The 5 item version of extrinsic effort had a Chronbach alpha 0.85. Extrinsic effort items included constant time pressures, interruptions and disturbances at work, job responsibility, pressured to work overtime and increasing demands of the job. The percentage of workers characterised into each exposure group across each need for recovery measure was first analysed. Each worker was classified into one of four age groups (17–29, 30–39, 40–49, 50–59). The age grouping was selected to allow for equal number within each group. The odds ratio and 95% confidence interval was then calculated for each potential risk factor and compared across each measure of need for recovery using the Mantel Haenszel Common Odds Ratio Summary statistic stratified across each of the 4 age groups. The stratum specific odds ratios and 95% confidence intervals were then calculated for each age stratum to investigate potential effect modification. Data were analysed using SPSS version 15 and Epi Info version 3.5.1.
Results There was a similar distribution of subjects across exposure groups at each time point (T1 and T2). The odds ratios for each risk factor across each time point was similar, as shown in table 1. The psychological demands were the strongest risk factors followed by physically demanding work and heavy lifting. Repetitive arm movement was also a risk factor but prolonged keyboard use was significantly weaker then the other physical demand variables and it is tentative as to whether the increase in the risk observed was a true effect. The risk from long working hours was similar to the risk for heavy lifting. There was a weaker odds ratio for late night or shift work but the results suggest that there was a true effect. Workers were approximately three times more likely to report a higher need recovery after work if exposed to psychologically demanding work, as measured using psychological job demands and extrinsic effort variables. Workers were approximately two times more likely to report a higher need for recovery after work when exposed to physically demanding job. Tasks requiring pushing and pulling, lifting loads greater than 16 kg, driving for more than half a day and performing almost continuous arm movement had odds ratios between 1.27 and 1.44. Gender did not increase the risk for need for recovery after work. Table 2 shows the risk from individual characteristics, work organisation and work content across each age stratum and whether the same effect was observed at
192
J.J. Devereux & L.W. Rydstedt
Table 1.
Risk factors for need for recovery at time 1 and time 2 adjusted for age.
Factor and exposure level
MH Common OR & 95% CI at T1
MH Common OR & 95% CI at T2
Individual characteristics Gender Men vs woman
1.13 (0.10–0.98)
1.08 (0.90–1.30)
Work organisation 42 or more hours worked per week Late or night shift vs day shift
1.79 (1.54–2.08) 1.28 (1.09–1.51)
1.67 (1.38–2.01) 1.46 (1.19–1.79)
Work content Psychological job demands Extrinsic effort Extrinsic effort without physical loading
3.10 (2.64–3.63) 3.22 (2.72–3.82) 3.14 (2.70–3.65)
2.91 (2.37–3.56) 3.14 (2.57–3.84) 2.57 (2.14–3.08)
1.94 (1.66–2.27) 1.43 (1.23–1.66) 1.30 (1.04–1.64) 1.27 (1.09–1.49) 1.68 (1.35–2.08) 1.33 (1.05–1.68) 1.19 (1.03–1.39)
1.98 (1.64–2.40) 1.41 (1.17–1.70) 1.26 (0.94–1.70) 1.27 (1.05–1.54) 1.44 (1.10–1.89) 1.33 (0.99–1.79) 1.07 (0.89–1.29)
1.50 (1.26–1.79)
1.44 (1.16–1.80)
Physical work demands Physically demanding job Task requiring pushing/pulling Lifting 6–15 kg Lifting 16–45 kg Lifting >45 kg Driving > half working day Using a keyboard for 4 or more hours a day Repetitive arm work Almost continuous arm movement vs Intermittent arm use and Regular arm movement with some pauses OR, Odds Ratio; CI, confidence interval
both time 1 and time 2 measurement of need for recovery after work. Table 3 shows the risk from physical work demands. The odds ratio for psychological job demands, extrinsic effort, driving, and tasks requiring pushing and pulling increased from the youngest to the oldest age group. The oldest age group had a much higher odds ratio compared to the youngest age group for the variables involving lifting greater than 16 kg and repetitive work. No such effect modification was observed for gender, working late or night shifts, lifting 6–15 kg, or using a keyboard. Prolonged working hours show an increased risk for the two older age groups compared to the two youngest age groups but this effect was not observed at both time 1 and time 2. The excess risk for the work content variables and the physical demand work variables was more than 30% in the older age group compared to the youngest age group.
Discussion Psychologically and physically demanding work increased the risk for need for recovery from work. Psychologically demanding work was the stronger risk factor
1.53 1.70
Work organisation 42 or more hours worked per week Late or night shift vs day shift 1.78 1.51 1.28
1.04 1.13
0.88
3.85 3.58 2.67
2.24 2.57
1.79
2.45 2.69 2.78
1.50 1.73
1.13
30–39
1.60 1.82 1.91
1.04 1.16
0.78
3.74 3.98 4.04
2.17 2.57
1.62
2.60 3.30 2.68
1.68 1.31
1.06
40–49
OR, Odds Ratio; CI, confidence interval X X This means that a: there is an increased risk in the oldest compared to the youngest group and, b: there is effect modification at both T1 and T2, defined by an excess risk of 30% or more.
2.62 2.32 1.85
1.25
Individual characterisitics Gender Men vs woman
Work content Psychological job demands Extrinsic effort Extrinsic effort without physical loading
17–29
OR (95% CI)
1.79 2.26 1.90
1.19 0.90
0.75
3.76 4.82 3.77
2.37 1.90
1.49
4.79 4.67 3.21
2.04 1.14
0.89
50–69
2.98 3.09 2.18
1.38 0.71
0.60
7.69 7.07 4.75
3.02 1.81
1.32
X X X
T1
Effect modification of age on the odds ratios for individual characteristics, work organisation and work content.
Factor and exposure level
Table 2.
X X X
X
T2
Does older workforce with high work demands need more recovery 193
2.07 1.300 1.157 1.302 1.330 1.064 1.176 1.351
Physical work demands Physically demanding job Task requiring pushing/pulling Lifting 6–15 kg Lifting 16–45 kg Lifting >45 kg Driving > half working day Using a keyboard for 4or more hours a day Repetitive arm work 1.38 0.89 0.63 0.87 0.76 0.59 0.81 0.87
3.11 1.90 2.13 1.95 2.34 1.92 1.70 2.11
1.81 1.23 1.46 0.95 0.99 1.18 1.16 1.21
30–39 1.23 0.85 0.79 0.64 0.57 0.67 0.80 0.78
2.66 1.80 2.70 1.41 1.71 2.09 1.68 1.86
1.78 1.29 1.35 1.26 1.28 1.33 1.23 1.62
40–49
OR, Odds Ratio; CI, confidence interval X X This means that a: there is an increased risk in the oldest compared to the youngest group and, b: there is effect modification at both T1 and T2, defined by an excess risk of 30% or more.
17–29
Factor and exposure level
OR (95% CI)
1.24 0.91 0.77 0.87 0.75 0.74 0.87 1.05
2.54 1.83 2.36 1.81 2.19 2.38 1.74 2.50
2.39 1.99 1.10 1.70 2.53 2.01 0.72 1.64
50–69
Table 3. Effect modification of age on the odds ratios for physical work demands.
1.63 1.35 0.60 1.14 1.47 1.08 0.47 1.06
3.51 2.92 2.02 2.53 4.35 3.75 1.09 2.54
X X X X X X
X X X X
T2 X X
T1
194 J.J. Devereux & L.W. Rydstedt
Does older workforce with high work demands need more recovery
195
compared to high physical work demands. Gender differences and prolonged keyboard work were not important factors. A comparison of the results measured at time 1 and time 2 shows similar effect sizes and so provides stronger support for the observed relationships. These results are supported by the cross-sectional results from Sluiter et al., (2003). They found that physical work demands were associated with need for recovery from work in construction, and hospital nurses. Our study also examined age as a potential effect modifier and found that older workers are more likely to have a higher need for recovery from work after performing psychologically and physically demanding work compared to younger workers. The excess risk for workers greater than 50 years old performing frequent lifting of loads >45 kg and experiencing high psychological demands was significant compared to the 40-49 year old age group. A high number of hours worked showed a marked increase in the likelihood for older workers at time 2 but not at time 1 so there is weaker evidence for this work organisation factor. The low response rate to the baseline questionnaire may have underestimated the effect size of work content variables, as a sub-sample of non-respondents indicated a higher time pressures compared to respondents (a potential response bias). Furthermore, the population may also be a survivor population in that older workers performing heavy lifting may have already left the study population before the baseline survey was conducted. This would make it harder to detect a true effect. Despite these limitations, an increased risk was observed for psychological and physical work indicating a true effect. There were no differences in age and gender in non-respondents confirmed by company records. Physical and psychological work demands were relatively stable over the follow-up period according to a re-survey of a cohort sub-sample. The majority of workers were unlikely to have changed from low to high exposure or vice versa. The need for recovery scale has shown to have good to excellent test-retest reliability under stable work conditions. No other study had examined age as an effect modifier across physical and psychosocial work demands so it is difficult to draw any firm conclusions. Further research is needed to explore this potential relationship as it is important for developing better health prevention strategies in the workplace. In this study a high need for recovery from work at time 1 was associated with taking more than 5 days off for health reasons in the last 12 months. (OR 1.62 95% CI 1.33–1.98) compared to workers taking no days off. Psychological job demands have been shown to be a determinant of long term mental strain in some occupational groups (Rydstedt et al., 2007) and the physiological effects of long term job strain can be detrimental to health (Rydstedt et al., 2008) Recovery is considered an important variable between adverse work demands, the development of work-related stress reactions and health problems in the long-term (Devereux & Rydstedt, 2008; Sluiter et al., 2003). It is important that the physical and psychological demands of work are assessed and controlled especially for older workers. Work organisations should consider
196
J.J. Devereux & L.W. Rydstedt
working together with older employees in their workforce to determine optimal workload levels so that the need for recovery from work is minimised. Furthermore, design changes to the work environment to support declining cognitive, perceptual and psychomotor abilities are recommended (Charness, 2008). Sluiter (2006) has suggested job specific functional tests for high demanding jobs in order to monitor individual functioning and facilitate appropriate interventions. We would like to thank the Health and Safety Executive for funding this study.
References Charness, N. 2008, Aging and human performance. Human Factors, 50, 548–555. de Croon, E. M., Sluiter, J. K., & Frings-Dresen, M. H. W. (2006). Psychometric properties of the Need for Recovery after work scale: test-retest reliability and sensitivity to detect change. Occupational and Environmental Medicine, 63, 202–206. Devereux, J. & Rydstedt, L. 2008, The relationship between psychosocial work factors and job stress: prospective data from the Stress and MSD Study. In L.Sznelwar, F. L. Mascia, & U. B. Montendo (Eds.), Human Factors in Organizational Design and Management-IX (IEA press, Santa Monica), 211–216. Devereux, J., Rydstedt, L., Kelly, V., Weston, P., & Buckle, P. 2004, The role of work stress and psychological factors in the development of musculoskeletal disorders: The Stress and MSD Study, (HSE Books, Sudbury). Rydstedt, L., Cropley, M., Devereux, J. J., & Michalianou, G. 2008, The relationship between long-term job strain and morning and evening saliva cortisol secretion among white-collar workers. Journal of Occupational Health Psychology, 13, 105–113. Rydstedt, L., Devereux, J. J., & Sverke, M. 2007, Comparing and combining the demand-control-support model and the effort-reward imbalance model to predict long-term mental strain. European Journal of Work and Organizational Psychology, 16, 261–278. Sluiter, J. K., de Croon, E. M., Meijman, T. F., & Frings-Dresen, M. H. W. 2003, Need for recovery from work related fatigue and its role in the development and prediction of subjective health complaints. Occupational and Environmental Medicine, 60, 621–670. Sluiter, J. K., Frings-Dresen, M. H. W., Van der Beek, A. J., & Meijman, T. F. 2001, The relation between work-induced neuroendocrine reactivity and recovery, subjective need for recovery, and health status. Journal of Psychosomatic Research, 50, 29–37. Sluiter, J. K. 2006, High-demand jobs: age-related diversity in work ability? Applied Ergonomics, 37, 429–440. van Amelsvoort, L. G. P. M., Kant, I. J., Bültmann, U., & Swaen, G. M. H. 2003, Need for recovery after work and the subsequent risk of cardiovascular disease in a working population. Occupational and Environmental Medicine, 60, i83–i87.
HUMAN ADVISORY
THE ROLE OF THE HUMAN IN FUTURE SYSTEMS – CONSIDERATIONS AND CONCERNS M.S. Young Human-Centred Design Institute, Brunel University, UK In 1990, Don Norman espoused his critical view of automation – namely, that its problems lie in being both too smart and yet not smart enough. Nearly 20 years on, we are no nearer to the ‘autopian’ vision of the future, although the ergonomics perspective on the problem has changed. In this paper, traditional frameworks and critiques of automation (allocation of function, ironies of automation) are summarised and contrasted against more contemporary thinking on the topic (support, cooperation and teamwork). While we wait for technology to catch up and for automation to be smart enough to be an effective member of a human-machine team, a new approach to automation is proposed based on principles of crew resource management. The argument is brought full circle by suggesting that ergonomics can take a lead on technology by specifying the capabilities of automation to fulfil the requirements of the team.
Automation – the ghost in the machine Technological progress is an inexorable tide – the rise of ubiquitous computing purports to make our lives safer and more efficient. From automatic protection systems in aircraft to artificial intelligence in our washing machines, the aims of such developments are very much in line with those of ergonomics – to improve safety, efficiency, and satisfaction. Nevertheless, the ergonomics literature on automation holds a much more conservative view, and we have known for some time now that to simply try to automate the human out of the loop does not provide the solutions that engineers crave. In the best-selling book ‘Jurassic Park’ (Crichton, 1980), the mathematician character Ian Malcolm criticises scientists for focusing too much on whether they can do something without stopping to consider whether they should. The same criticism has been levelled at designers of automation by some of the most influential ergonomists working in this field (e.g., Parasuraman, 1987; Wiener and Curry, 1980). Whilst the principle of avoiding technology merely for its own sake remains valid, it is fair to say that the current view in ergonomics is somewhat more mature in trying to understand how humans and automation can work together safely and effectively. It would be foolish to think we could stem technological progress – nor would we want to, lest Ian Malcolm accuse us of the kind of narrow-minded thinking he refers to as ‘thintelligence’. Humans and technology can and indeed should work together, and technological progress should be exploited – but in the 199
200
M.S. Young
right way. This paper considers the future for the ‘ghost in the machine’, arguing for a philosophical approach to the design of automated systems which allows humans and technology to coexist in a truly dualist system.
Promises and problems Before looking to the future, it is worth briefly reviewing some key developments in ergonomic thinking on automation. The classical perspective is captured in three seminal works. Firstly, Lisanne Bainbridge described the ‘ironies of automation’ in 1983. An initial irony lies in the designer’s view of the human operator as being unreliable or inefficient. Automation is popularly assumed to circumvent human error – by simply removing the human element in the system. What designers overlook, though, is that they are human too – and shortcomings in the system design can cause new problems. For instance, the new European Rail Traffic Management System (ERTMS) offers unprecedented levels of train protection, and to all intents and purposes will be an advance in performance and capacity on the rail network. However, its automatic protection systems are dependent on a set of data input by the driver at the start of a journey – thus moving the human error potential further back in the causal chain. Furthermore, Bainbridge argues, the operator is often left to do the tasks which the designer cannot solve through automation. But in order to take over control, a human operator must be practised at the task – which is impossible when the automation has been controlling it. Operator training cannot offer a solution, since it is impossible to train for the unforeseeable, so only general strategies may be learned. The irony is then in training the operator to follow instructions yet expecting them to provide intelligence in the system. Perhaps the final irony, Bainbridge notes, is that it is the most successful automated systems, with little need for human intervention, which require the greatest investment in operator training. James Reason (1990) expands on this with his ‘catch-22’ of human supervisory control, in that humans are only present in an automated system to deal with emergencies. They do this by drawing on stored knowledge of such, but with little opportunity to practice procedural responses – coupled with the unique nature of each emergency – means they have little knowledge or experience to draw upon. Reason concludes the catch-22 by stating that automation is most effective when it is least required, and vice-versa. That is to say, under normal operating conditions, automated systems can cope perfectly well. However, so do humans – thus begging the question of why automation has been implemented in the first place. In emergency situations, stressed humans can become overloaded and performance can deteriorate; it is under such circumstances when assistance would be most valuable. Ironically, it is in these situations when automation also surrenders, relying on the human to provide creativity and quell the emergency. Also in 1990, Donald Norman espoused a view that suggested it is not the presence of automation per se which is the problem, rather that the design of automated systems is inappropriate. The problem with automation is that it is at an intermediate level of intelligence – specifically providing an insufficient level of feedback. This
The role of the human in future systems
201
imposes more cognitive demands (the need to actively gather information about system state) and fewer perceptual-motor demands. Ironically, it is exactly these perceptual-motor tasks which humans are most skilful at, rather than the cognitive tasks which generally require greater mental demands. Thus the culprit is not necessarily the automation, rather the lack of feedback and interaction which keeps the human out of the loop. All of this leads Norman to a new irony: that it is not powerful enough. If automation were perfect (i.e., never fails), feedback would be unnecessary. However, the vision of a perfect automation system is still a long way off; the unique skills and flexibility of human operators will continue to prove crucial in critical situations. Take the example of United Airlines Flight 232, on 20th July 1989 (as reported by Faith, 1996). The DC-10 aircraft had suffered an explosive failure of its number two engine (the DC-10 has three engines, one on each wing and one mounted on the tail structure; it was the latter engine which failed), which in the process had destroyed the hydraulics to the control surfaces. Unable to fly the aircraft by conventional means, the crew enlisted the help of an off-duty pilot who happened to be on board, and they managed to fly to an airport at Sioux City, Iowa, purely by using the balance of the two remaining engines. The approach was going well considering such improvisation, but an unfortunate windshear on landing caused a catastrophic impact with the loss of 112 lives. However, the fact that there were 184 survivors can be put down to the ingenuity and skill of the flight crew. The common issue running through the critiques of Bainbridge, Norman and Reason is that automation changes the role of the human in the system from active control to passive monitoring – a task at which humans are particularly poor (Parasuraman, 1987). This out-of-the-loop performance problem can be manifest in vigilance failures (Molloy and Parasuraman, 1996) or difficulties recovering control from automation failure (Kaber and Endsley, 1997). Such problems have been attributed to mental workload (Young and Stanton, 2002), situation awareness (Endsley, 1995) and trust (Parasuraman and Riley, 1997). Although there may still be controversy surrounding the cause of automation-related complacency, there is no doubt that monitoring performance is impaired if the task is executed by automation instead of a human (Molloy and Parasuraman, 1996) – it is the qualitative change in the nature of the task that causes the problem. However, more recent thinking supplants this view, suggesting that whilst automation does change the nature of work, it actually adds a whole set of new demands. The problem of automation now is one of coordination.
Teamwork and taskwork In the last 10 years, there has been a shift in emphasis across the ergonomics literature regarding the interaction of humans and automation – and the associated solutions for the problems of automation. Rather than thinking in terms of the passive monitor, the human’s role is now seen as active manager of a set of resources, which happens to include the automated system (Dekker, 2004). The human is no longer out of the loop, just more distanced from it (Schutte, 1999). Automation does
202
M.S. Young
indeed qualitatively change the nature of the task, but this is by adding a whole new set of coordination demands (Parasuraman et al., 2000). Such a view is not wholly inconsistent with previous observations – one of the consequences of being a manager is that the demands increase at crucial times such as abnormal or emergency scenarios. But instead of this being due to vigilance monitoring issues, it is now felt that the problem arises from the increased demands of teamwork and taskwork. Teamwork is defined as “the ability of team members to work together, communicate effectively, anticipate and meet each other’s demands, and inspire confidence, resulting in a coordinated collective action” (Cannon-Bowers and Salas, 1997). In the teamwork literature, the demands of teamworking are divided into taskwork (those task-oriented activities which are necessary to actually get the job done) and teamwork (the coordination and communication activities, over and above taskwork, which are required to work together effectively). Thus in some cases, adding more members to the team can actually make overall performance less efficient, since the additional demands of teamwork outweigh the reductions in taskwork. Whilst the teamworking literature typically considers human-human teams, the principles of communication, cooperation and coordination apply across humanmachine teams when it comes to automation. Parasuraman and Riley (1997) defined automation as “the execution by a machine agent (usually a computer) of a function that was previously carried out by a human” (p. 231). Traditionally, this has been thought of as taking tasks piecemeal away from a single human operator, but it could equally be seen as the machines replacing human agents in the team. In essence, then, we are talking about delegation of tasks to another agent (cf. Dekker, 2004). But in doing so, we must ensure that the teamworking elements – the communication, cooperation, and coordination – remain intact. In the abnormal scenario, for instance, the human needs support from the automation in order to resolve the situation. The problem now is that most automated systems are not designed to behave like humans when it comes to teamwork. Take Norman’s (1990) case study of the Boeing 747 with the loss of engine power. In this example, an autopilot system attempted to compensate for the loss of power by balancing the control surfaces (almost an inverse of the Sioux City incident described above). However, as Norman rightly points out, the autopilot did not provide feedback to the flight crew on its actions – so when it could compensate no more, the human team members were faced with a drastically worse situation than if they had been informed earlier. From the teamworking perspective, it is the communication which is missing.
Complemation and common frames of reference Any good team is built up of members with complementary skills, such that the task demands can be met and the team’s goals achieved. If any of the team members happen to be machine, the situation should be no different. Schutte (1999) coined the term ‘complemation’ to sum up the principle of exploiting automation to enhance
The role of the human in future systems
203
human capabilities while compensating for their limitations. For instance, we could allow the human to provide creativity and adaptability, while letting the machine store information and make precise measurements – maximising the strengths of each team member. Such an approach is a good start in getting humans and automation to work together more effectively, and it has the added advantage of avoiding the potential ‘technophobic’ stance of previous views. We know enough about human performance to delineate the strengths of the human component in this team, and technological progress is improving computing power all the time. Nevertheless, this only really covers the taskwork element of the team – we still need to solve the teamworking aspect of the equation. A high-profile air crash involving a new generation of automated aircraft exemplifies the teamwork problem. On 26th June 1988, the new Airbus A320 was being demonstrated at an air show in France. The pilot was making a low pass with the gear extended, but on attempting to pull out at the end of the circuit, he found that power was not available, and the aircraft crashed into trees at the end of the runway. Explanations differ according to the cause of this accident, but the automation was implicated as the pilot fought the aircraft for control. It has been said of situations such as this that instead of fighting the computer, pilots should occasionally switch it off and look out of the window. Dekker (2004) argues that we should move on from this point of view, and instead of switching it off, we should be asking ‘how do we get along together?’ For automation to be an effective team player, it needs to support coordination and cooperation, which means the human and the machine understanding each other’s perspective as well as their abilities and limitations. Hoc et al. (in press) refer to a ‘common frame of reference’ in which the human and machine understand each other’s goals and work in tandem to achieve them, but this crucially depends on information constantly flowing between the agents (cf. Griffin and Young, 2008). The essential basis of coordination and cooperation, then, is communication. This echoes Norman’s point that it is all about feedback, but the difference is that information flows in both directions. Rather than considering more or less automation, then, coordination is all about the quality of the automation, not the quantity. In order to understand what high quality automation consists of in the teamworking context, we need to revisit some frameworks of automation.
Taxonomies and tasks Up until relatively recently, approaches to classifying automation in the ergonomics literature have focused on task-based divisions of labour. The original Fitts’ list, devised in 1951, specifies those activities for which human skills surpass those of machines, and vice-versa. Later developments put forward taxonomies based on levels of authority, to decide how the tasks are divided (e.g., Sheridan and Verplanck, 1978). More recently a similar notion has dichotomised the authority into ‘hard’ and ‘soft’ automation, depending on whether the machine or the automation has ultimate control over task decisions (see e.g., Young et al., 2007). Such models
204
M.S. Young
for determining allocation of function dominated for decades, until the likes of Bainbridge and Reason shared their realisations about the impact of such fixed demarcation of tasks on human performance. The landmark of Sheridan and Verplanck’s (1978) taxonomy was in the recognition that automation is not an all-or-none option, and that by exploiting different levels of automation it is possible to assist the operator rather than replace them (Kaber and Endsley, 1997). Thus the general consensus moved on to consider adaptive automation – intelligent systems which monitor the environment and regulate the level of automation to maintain an optimal state for the operator (Byrne and Parasuraman, 1996). Whilst adaptive systems may be a laudable objective, the practicalities of transferring control and the technological barriers have so far proved insurmountable. Thus adaptive automation has fallen out of favour somewhat, to be replaced by the more contemporary concepts of operator support (e.g., Young et al., 2007). Nevertheless, the work of Sheridan and Verplanck (1978) is still proving inspirational to the latest generation of automation frameworks, which do focus on operator support from more of a teamworking perspective. Parasuraman et al., (2000) redefined the levels of automation matrix to consider the action cycle across the task – from information acquisition, analysis, decision and action selection, to action implementation. It is noted that the continuum of manual-automatic levels of authority can be applied across each of these task stages to determine the optimal balance of automation across the whole task. Hoc (2001; Hoc et al., in press) took this approach a step further in considering more explicitly the aspect of human-machine cooperation to propose a more teamwork-based allocation of function model. Hoc (ibid.) talks in terms of managing interference between the human and the machine, working together to establish that ‘common frame of reference’. The model proposes three levels of cooperation and four modes of task automation in a matrix of human-automation interaction. At the meta level, human and machine establish models of each other’s operation and behaviour, in order to provide a platform for coordination. The plan level defines rules of engagement based on task context, and criteria for automatic support or task delegation. At the action level, the behaviours and rules are implemented according to the demands of the task. The modes of operation somewhat reflect a hybrid of Sheridan and Verplanck’s (1978) taxonomy and the information processing cycle in the model of Parasuraman et al. (2000), covering perception, mutual control, function delegation, and full automation. The crucial distinction in Hoc’s model is the establishment of this common frame of reference – essentially a mental model of system operation, but one which must be held by both human and machine about the other’s behaviour. One of the key aspects of this is the awareness of context and intent – vital in interpreting the actions implemented in a given situation. Hoc et al. (in press) provide the example of a lane-keeping system in cars – which warns the driver when straying out of his/her lane. Clearly, sometimes this activity is legitimate – when overtaking, for example. The system has access to the vehicle’s electronics, though, and so only provides a warning if the driver is moving across the lane markings without having used a turn signal. Whilst this is a crude rule, it illustrates the importance of intent
The role of the human in future systems
205
and context in maintaining that common frame of reference and hence the smooth dynamics of the team. Although the concept of operator support is taking hold, it could be argued that in many cases designers pay it mere lip-service, skirting round the ideals of teamwork and cooperation. Staying with the example of vehicle automation, many systems are ostensibly designed to support the driver (indeed, this exonerates manufacturers from any potential litigation, since drivers are always held to be responsible), but in most cases these fall into the old trap of simply automating what can be automated. Once again, the driver has to manage the automation as well as drive the car, rather than working with the system on a common goal. Young et al. (2007) capture the principle by suggesting that true driver support should act as a human co-driver – providing advice when needed, assistance when necessary, but largely remaining invisible in the background under normal conditions. This philosophy fosters human skills for the task, and in cases where automation does have to intervene, it will communicate efficiently with the driver and work on the basis of the task context and driver’s intentions. Needless to say, technology undoubtedly has some way to go before automation can be smart enough to know what the human wants to do. In the meantime, we can set out a new philosophy for design and implementation of automated systems based on the contemporary principles of cooperation, communication and coordination.
Re-training the ghost Thus far we have seen how traditional perspectives on automation – which view the human as a passive monitor and divide the task piecemeal according to fixed rules – have been superseded by a more flexible approach based on the human and machine working as a team. In designing optimal human-human teams, the aims are to have a balance of skills and good communication and understanding between team members. Likewise for human-machine teams, the ultimate objective would be to design an automated system with complementary taskwork skills (cf. Schutte, 1999) and good teamworking abilities (cf. Hoc, 2001). The former is not so far removed from a Fitts’ list type approach, and technology is increasing the range of skills available with automation (indeed, it is high time the Fitts’ list was updated). As far as teamwork is concerned, though, the technological barriers are much higher. Training and design are two sides of the same coin, something that Rigner and Dekker (2000) realise in considering the impacts of flight deck automation on crew resource management (CRM). CRM is essentially a teamwork programme which has become a key component of flight crew training (Wiener et al., 1993). As Rigner and Dekker (2000) point out, automation affects the dynamics on the flight deck and needs to be properly integrated into the training programme – CRM should focus on utilising automation as a resource, and coordinating its input to the task as a whole. This sounds promising from our teamwork perspective, but perhaps does not accord with the guiding philosophies of ergonomics since it relies on fitting the
206
M.S. Young
human to the task. Young et al. (2007) argue for taking the concept a step further, applying the principles of CRM training to the design of automated systems. This may be an interim measure until technology allows us to fully integrate automation into the team as described above, and it is envisaged here that CRM-designed automation can be achieved in two ways. Firstly, by erring on the side of ‘soft’ automation, thus leaving the human in active control and able to delegate tasks as appropriate – in line with the frameworks proposed by Parasuraman et al. (2000) and Hoc (2001). Secondly, as argued earlier in this paper, the teamworking aspect ultimately comes down to communication in both directions – which means a significant design effort on the control-display interface to optimise the flow of information (cf. Griffin and Young, 2008). As the ‘ghost in the machine’, automation can sometimes be seen as having a mind of its own – a problem which is all too evident in some of the incidents described earlier in this paper. A teamworking approach, using the principles of CRM to emphasise coordination, cooperation and communication, can serve to overcome these problems in human-automation interaction and lead us forward to a more ‘autopian’ ideal (cf. Hancock et al., 1996). Bringing the arguments full circle, this is not about a technophobic reluctance to automate – in fact, it is quite the opposite. Both Norman (1990) and Dekker (2004) would agree that automation has been ‘thintelligent’, and we could actually see a case for ergonomics driving technological progress, rather than the other way around. If we are to design automation to fit with a human team member, we need to reverse Fitts’ list and define those areas of human weakness where technological support is necessary – for example, hazard perception in driving. This will require a much greater effort than simply automating what we can, but it enables us to define how we should. Where Wiener and Curry (1980) argued that automation had already passed its optimal point, instead it seems the best is yet to come.
References Bainbridge, L. 1983, Ironies of automation, Automatica, 19(6), 775–779. Byrne, E. A., and Parasuraman, R. 1996, Psychophysiology and adaptive automation. Biological Psychology, 42, 249–268. Cannon-Bowers, J. A., and Salas, E. 1997, Teamwork competencies: the interaction of team member knowledge skills and attitudes. In O. F. O’Neil (ed.), Workforce readiness: Competencies and assessment (Erlbaum, Hillsdale, NJ), 151–174. Crichton, M. 1990, Jurassic Park (Knopf, New York). Dekker, S. 2004, On the other side of promise: what should we automate today? In D. Harris (ed.), Human factors for civil flight deck design (Ashgate, Aldershot), 183–198. Endsley, M. R. 1995, Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. Faith, N. 1996, Black box: the air crash detectives – why air safety is no accident (Boxtree, London).
The role of the human in future systems
207
Griffin, T., andYoung, M. S. 2008, The flow of information. Clued Up (2008 Issue), 62–67. Hancock, P. A., Parasuraman, R., and Byrne, E. A. 1996, Driver-centred issues in advanced automation for motor vehicles. In R. Parasuraman and M. Mouloua (eds.), Automation and human performance: Theory and applications (Lawrence Erlbaum Associates, Mahwah, NJ), 337–364. Hoc, J. M. 2001, Towards a cognitive approach to human-machine cooperation in dynamic situations, International Journal of Human-Computer Studies, 54, 509–540. Hoc, J. M., Young, M. S., and Blosseville, J.-M. in press, Cooperation between drivers and automation: Implications for safety, Theoretical Issues in Ergonomics Science. Kaber, D. B., and Endsley, M. R. 1997, Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety, Process Safety Progress, 16(3), 126–131. Molloy, R., and Parasuraman, R. 1996, Monitoring an automated system for a single failure: vigilance and task complexity effects. Human Factors, 38(2), 311–322. Norman, D. A. 1990, The ‘problem’ with automation: inappropriate feedback and interaction, not ‘over-automation’, Phil. Trans. R. Soc. London B, 327, 585–593. Parasuraman, R. 1987, Human-computer monitoring, Human Factors, 29, 695–706. Parasuraman, R., and Riley, V. 1997, Humans and automation: use, misuse, disuse, abuse, Human Factors, 39(2), 230–253. Parasuraman, R., Sheridan, T. B., and Wickens, C. D. 2000, A model for types and levels of human interaction with automation, IEEE Transactions on Systems, Man and Cybernetics – Part A: Systems and Humans, 30(3), 286–297. Reason, J. T. 1990, Human Error (Cambridge University Press, Cambridge). Rigner, J., and Dekker, S. 2000, Sharing the burden of flight deck automation training, International Journal of Aviation Psychology, 10(4), 317–326. Schutte, P. 1999, Complemation: an alternative to automation, Journal of Information Technology Impact, 1(3), 113–118. Sheridan, T.B., and Verplanck, W.L. 1978, Human and computer control of undersea teleoperators (MIT Man-Machine Systems Laboratory Report, Cambridge, MA). Wiener, E. L., and Curry, R. E. 1980, Flight-deck automation: promises and problems. Ergonomics, 23(10), 995–1011. Wiener, E. L., Kanki, B. G., and Helmreich, R. L. 1993, Cockpit resource management (Academic Press, San Diego). Young, M. S., and Stanton, N. A. 2002, Attention and automation: new perspectives on mental underload and performance, Theoretical Issues in Ergonomics Science, 3(2), 178–194. Young, M. S., Stanton, N. A., and Harris, D. 2007, Driving automation: learning from aviation about design philosophies, International Journal of Vehicle Design, 45(3), 323–338.
AUTOMATION AND TECHNOLOGY IN 21ST CENTURY WORK AND LIFE S. Sharples Human Factors Research Group, Faculty of Engineering, University of Nottingham, Nottingham NG7 2RD, UK Technology is pervasive in 21st century life. Many types of automation technology change the ways in which we complete tasks in our work and home. Bainbridge (1983) identified a set of “ironies of automation”. These ironies were identified in the context of process control operations, but we now encounter automated systems in a wide variety of contexts, ranging through the home and work environments. Users of automation also often have a degree of choice as to the extent to which they use the automation available. This paper analyses the impact of automation in three different case examples – rail signalling automation, handheld computing for pedestrian navigation, and word processing interfaces, and considers the relevance of three of Bainbridge’s ironies to modern automation. A new irony, that addresses the need for the user to choose to seek to actively understand how the automation works in order to fully reap its benefits, despite the aim of that automation potentially being to actually conceal the complexity of its workings, is proposed for consideration.
Ironies of automation Bainbridge (1983) wrote a landmark paper that identified a set of ironies of automation – based around the consequence that “the more advanced a control system is, so the more crucial may be the contribution of the human operator”. These ironies covered the effect of automation on manual and cognitive skills, operator attitudes and considered both the impact of requiring an operator to monitor whether an automated system is working effectively, and the implications for the operator if they are required to intervene in control of the automated system. The nature of the proposed ironies were generally the unintended side-effects of automation that in many of Bainbridge’s examples actually potentially worsened, rather than improved, the situation for the operator and the effectiveness of the system as a whole. The systems primarily considered by Bainbridge were those that typically aimed to “replace human manual control, planning and problem solving by automatic devices and computers”. She acknowledges that even these highly automated systems still involve humans in a number of roles, including monitoring, maintenance and future development. The systems considered by Bainbridge primarily fall into higher level of automation, categorised for example by Parasuraman et al. (2000) as ranging from automatically executing actions and informing the human operator, to 208
Automation and technology in 21st century work and life
209
the computer deciding everything, acting autonomously and “ignoring the human”. More and more however we encounter automated technologies that rather than taking over task elements, advise or guide us on how to complete tasks, such as in navigation support systems or integrated automation within word processing applications. Critically, we also often have the opportunity to choose the manner and extent of use of the automation available. This paper specifically considers three of the ironies identified within Bainbridge’s landmark paper, and discusses their relevance to a set of example recent case applications. In addition, the paper discusses whether any new ironies are emerging with the use of current pervasive automated technologies. The three ironies of automation that will be specifically considered are (paraphrased from Bainbridge (1983): 1. A designer who tries to eliminate the operator still leaves the operator to do the tasks which the designer cannot think how to automate. 2. An automatic control system has been put in because it can do the job better than the operator, but the operator is being asked to monitor that it is working effectively 3. An operator will only be able to generate successful new strategies for unusual situations if he has an adequate knowledge of the process
Case 1: Railway signalling automation Rail signalling presents a particularly interesting context for examining automation due to the variety of technologies in use, ranging from manually controlled lever frames to VDU based automatic routing systems. Signalling systems that include automatic route setting technologies have been analysed in a number of different studies over the past ten years (e.g. Nichols et al., 2000; Balfe et al., 2007; Balfe et al., 2008). A typical VDU based signalling interface is shown in Figure 1. When Automatic Routing System (ARS) is used, the train routes are automatically set, based on timetabling information, and conflicts between two trains which are scheduled to travel on the same track resolved by conflict resolution algorithms. The route set by ARS is indicated by an area of the track in front of the train being highlighted, and signallers have the opportunity either to anticipate the route to be set by setting the route in advance of ARS, or by using functions within the system to force the automatic system to prioritise particular trains. When considering the ironies of automation, it is perhaps no longer fair to describe the elements of the task that are not automated as being those that the designer “cannot think” how to automate. Theoretically, if all train movements are timetabled and all information in the ARS system is up to date, the operator should be able to simply monitor the movements of the trains and not have to intervene. However, observations of and interviews with signallers (Balfe, in preparation) suggest that this is rarely or never the case. So why do operators still end up actively controlling the system? Reasons include that the system cannot automate all the train movements or the operator chooses to anticipate or override the automatic route setting. An example of this is that
210
S. Sharples
Figure 1. A VDU based rail signalling interface (black and white image of colour interface).
some train movements (e.g. movement of train stock into a depot) may not be timetabled, or trains may run so early or late that their timetabling information can no longer be used effectively. A signaller may also choose to anticipate ARS and set the route before the automatic system intervenes, or may manually block parts of track to force ARS to select an alternative, for reasons such as restrictions in track availability. An example of this is as follows: if a slow train is late running, the automatic system may hold up an express train in order to allow the slower train (scheduled to travel through that part of track at an earlier time) to pass. In fact, the signaller may prefer to allow the express train to travel onto the track first, as the slow train is already late, and this action presents the express train also experiencing a delay. This may only be achieved by a manual intervention with the system. This example also illustrates the relevance of the other two ironies – the automatic route setting is introduced to enable a single signaller to control a larger area of track, yet the signaller retains the responsibility to monitor that the route setting is working effectively (i.e. minimising delays on the area under control). Interviews conducted by Balfe (in preparation) particularly illustrate the relevance of the operator’s understanding of the automation technology and conflict resolution algorithms –signallers are not explicitly trained in ARS and instead learn “on the job”. Whilst some operators may report that they understand how the conflict resolution operates, reports of the rules themselves vary indicating a lack of transparency of the system and its underlying rules. One of the most striking things observed in visits to signal boxes however is the wide variation in attitudes towards and strategies used in completing the signalling task. All signallers have safety as an absolute priority and have confidence in the underlying interlocking that prevents conflicting routes being set. However, detailed observations of signaller activities (e.g. Nichols et al., 2000; Balfe et al., 2008) illustrate that different signallers at the same workstation will vary in the extent to which they manually set routes. Signallers appear to differ in the extent to which
Automation and technology in 21st century work and life
211
they allow ARS to “take control” – some try to anticipate all routes and “beat” ARS to setting the routes – others are happy to monitor and only intervene when they wish or need to. Signallers do have the opportunity to “query” ARS to identify why certain route settings have been made. The use of this facility, along with discussion with colleagues and on-the-job training, contributes to the signaller’s mental model of how the signalling automation works. This first case suggests that all three of the ironies under consideration do apply to automation in rail signalling and that, in addition, even in an example task that is quite close to the “process” tasks originally considered by Bainbridge, the individual has control over the way in which they use the automation to support their decision making. Rather than simply monitoring the process, the signaller has an opportunity to interact with the system and the automated support to work with the automation.
Case 2: Pedestrian geographical information support The second case of automation to be considered is pedestrian geographical information support. Work conducted by Nixon et al. (2008) has examined the way in which the design of handheld navigation devices affects navigation performance, and particularly confronts the challenge of displaying data on small screen displays. As display screens, batteries, processors and control interfaces are becoming smaller, the capability to integrate real time geographically linked information sources is becoming a reality; yet we still do not have consistent methods of presenting integrated spatial and non-spatial information in order to effectively support navigation tasks. Work with the fire service has identified potential for providing valuable support in safety critical environments, such as indicating presence of gas mains on a CAD diagram of a building to be evacuated, accompanied by updated registers of those expected to be working in the building at the time of evacuation. We are consistently presented with the challenge of presenting increasingly complex data in a shortened time frame on a smaller screen – if this challenge can be overcome then vital automated support can be provided in a safety critical environment. A prototype interface to be used in a pedestrian navigation guidance system is shown in figure 2. If actively implemented this system would use positioning technology to locate both the position and orientation of the user, and could work on the same route setting basis as an in-car system, where the user inputs their required destination, and the system automatically calculates the route needed to get to that destination most efficiently. If used in firefighting, theoretically this could be integrated with information about fire exits, or even predict the likely spread of fire or smoke. Such technology is a good example of automation of a task that has previously been conducted using a variety of artefacts in the real world, including maps or plans, local knowledge or even expectations of building or road layout based on previous experience of similar environments. The task of pedestrian navigation combines the use of knowledge in the world and knowledge in the head (Norman, 1988), where the use of electronic handheld navigation support constitutes an element of the “knowledge in the world”. Rogers and Scaife (1999) describe the use of
212
S. Sharples
Figure 2. Three variations of a prototype handheld pedestrian navigation display (black and white image of colour interface) (Nixon et al., 2008).
such technology to support active decision making as “computational offloading” – signifying the use of technology to reduce the amount of cognitive effort needed to solve problems. Artman and Garbis (1998) also highlight the importance of use of real world artefacts in naturalistic decision making in emergency situations. In the example of pedestrian navigation support, all elements of the navigation task are automated, leaving the user to only complete the physical interaction with the environment and check on reaching the destination that they have in fact reached the correct location. Therefore the first irony being considered, that the tasks that are hard to automate are those that still remain, is not so relevant here. However, the operator is to some extent still required to monitor that the automation is working correctly. This particular task very much relies on the combination of information in the world, head and on the interface being used. Consider the example of using a device to locate an office in a large tower building. If the system directs you towards floor 12, but on arrival on floor 12 you are greeted by company insignia that you do not recognise, you may assume that you have been directed inappropriately. It is then up to the user to decide which information to trust and act upon – that in the world, or that on the interface. Finally, the irony related to generation of new strategies on the basis of adequate knowledge of the process can be considered to be relevant if we consider the “process” to refer to the broader context of development of geographical knowledge. Whilst Zimmer (2004) showed that users of fragmented maps developed as high a level of spatial knowledge as those presented with full maps, we know that if knowledge is in the world and we do not need or try to transfer it to our memory then we do not tend to do so. An illustration of this is in mobile phone use, where we do not tend to memorise phone numbers that are stored in the phone’s memory. Indeed, in the context of in-car navigation systems, Burnett & Lee (2005) found that “drivers who received typical vehicle navigation system instructions remembered fewer scenes, were less accurate in their ordering of images seen along routes and drew simpler maps which included a smaller number of landmarks, when compared
Automation and technology in 21st century work and life
213
to those using traditional methods.” Again, studies have shown large individual differences in the use of such systems, and also in the use of the landmarks in the real world to develop knowledge of the geographical layout of the environment. This second case particularly emphasises the continued relevance of the second and third ironies being considered – that an automatic control system has been put in because it can do the job better than the operator, but the operator is being asked to monitor that it is working effectively and that an operator will only be able to generate successful new strategies for unusual situations if he has an adequate knowledge of the process. The need for a broader view of the “process” is highlighted, with the general development of geographical knowledge or routes to be navigated being more appropriate for consideration. Again, rather than considering the user as a monitor of the system, the user is working with the system and other artefacts in order to construct knowledge and actively make decisions.
Case 3: Word processing interface support tools The final examples of automation to be considered are standard tools embedded with computer interfaces such as word processing. Within computer interfaces we are frequently offered different types of automated support, from automatic form fill-in, memory for passwords, graph drawing wizards, display only of frequently used menu items, automatic text formatting and paragraph spacings, as well as the Microsoft “paperclip”. However, as Lane et al. (2005) have pointed out, “even experienced users are inefficient in their use of graphical interfaces”. We frequently observe that these automated features are disabled or overridden, suggesting that they may not in fact be achieving their aim. Can the ironies of automation explain why? As we have witnessed applications such as Microsoft Word develop over the years, the level of automation has increased. However, there are still aspects of the word processing task that do not have automatic support. For example, Word 2003 has the potential to automatically recognise, and, if wished, correct some spelling mistakes, but it is not able to automatically check that the spelling of an individual’s name is correct if this name is not stored in the dictionary, and may actually inappropriately correct a spelling of a name that is very close to the spelling of something that is in the dictionary. In other words, “easy” spellings are automatically corrected, but “difficult” ones are not. The second irony, that the user is required to monitor that the automation is working correctly, also applies to word processing, and the added complexity here is that the third irony is also relevant – an adequate knowledge of the system is needed in order to develop new strategies. This is best illustrated by a separate example. If a user wishes to indent some text, then a person who is applying a “manual” metaphor will use the tab key, but one who is taking advantage of and embracing the application’s automatic functionality will select the “indent” icon or manipulate the paragraph format via the paragraph menu. As long as the space and paragraph characters are not visible, the layout will look the same, but the text might behave differently when edited (e.g. the formatting of one paragraph may influence the formatting of the subsequent added paragraph). In order to understand why
214
S. Sharples
the document is behaving in a certain way, and to avoid the presence of a form of latent error within the document, the user is required to have an appropriate mental model of the automatic formatting that has been applied. However, to develop the mental model they may have to either explore the document, by viewing paragraph and spacing marks, or analysing menu settings. But an action such as displaying paragraph and spacing marks results in a more cluttered display that may be more visually confusing and unpleasant, so users may be reluctant to take this route. Again, we see, and indeed many interfaces encourage and support, large individual differences in the way in which such applications are used. Individuals have the opportunity to choose their own menu layouts, visibility of paragraph markings, use of automatic support tools such as the Microsoft paperclip, and automatic formatting tools such as automatic spelling correction, formatting and heading identification.
Review of Ironies of Automation in current systems Whilst the wording of the ironies of automation relate to monitoring systems in general, the cases presented in this paper illustrate that the three ironies considered in detail are still relevant in current automation systems and can be applied to a broader context than process control. It may be appropriate to change the wording (paraphrased from the Bainbridge paper) from monitor to user, and from process to system. Indeed, the distinction between the user and the automation becomes blurred when we consider the overall process of decision making, and it is more appropriate to adopt a joint cognitive systems perspective (Hollnagel 2001). A key difference between the type of automation technology now being used from that considered by Bainbridge is the element of user choice – in most systems, ranging from setting our home video to dialling a number on our mobile phone, we have the opportunity to choose whether we use the automated approach or a more manual approach.
A new “irony”? However, the cases also suggest the need for a potential new irony of automation. The ironies proposed by Bainbridge that considered operator attitudes covered the issues of when a job is perceived as being “deskilled” by being reduced to monitoring, negative effect on operator attitudes, and that deskilled workers may insist on high pay level as remaining symbol of status which is no longer justified by job content. However, in addition to these issues, the cases above prompt the proposal of a possible new irony: Whilst automation may aim to “protect” the user from the complexity of the system, it is often only when a user enthusiastically explores the complexity of a system that they can often understand how the automation works, and thus harness its full potential. Automation is often only at its most effective when it is enthusiastically embraced and actively attended to by the operator. Automation is usually intended to reduce
Automation and technology in 21st century work and life
215
the workload of the operator, and possibly not require them to attend to all the information that they may have needed to use with a non-automated system. In practice a user can only extend their understanding of the system, general knowledge and skill, to understand how the automation works and to enable them to complete more complex or unusual tasks, by actively choosing to attend and work to understand how the automation works. This motivational element is critical to the success of automation, particularly in systems which are not essential to use, and may well enhance the user experience (e.g. by producing a more neatly formatted word document, avoiding an automatic route setting system rigidly applying rules, or taking a more efficient navigation path). Users may need to be willing to change the way they work, to go through a learning curve (at the very least by reading instructions or help files, exploring different menus or observing expert users) and may initially become worse at a task before they can take advantage of the automation. They may also need to choose to attend to information to which the system does not in fact require or expect them to attend or to actively work to understand the way in which the system works to allow them to form a useful mental model.
Summary of ironies applied to case examples The wording of the three original and new ironies can therefore be summarised as follows, to reflect the wider context of current automated systems: • The designer who tries to eliminate the user still leaves the user to do the tasks which the designer cannot think how to automate • An automatic control system has been put in because it can do the job better than the user, but yet the user is being asked to monitor that it is working effectively • A user will only be able to generate successful new strategies for unusual situations if he has an adequate knowledge of the system • Automation may only maintain or extend operator skills or knowledge when it is embraced and actively attended to by the operator If we are to design systems that support operations effectively, it may be appropriate to in fact consider only two types of system: fully automated (i.e. with no user at all) or one where the user is always in control, with automation confined to assistance with information capture and data handling in order to enhance the speed and accuracy of decision making. Any human involvement will always require the user to have a good model of the system, so if this is built into the normal operation of the system the user is likely to be more alert and satisfied with their system use.
Conclusion The cases examined in this paper suggest that the ironies of automation considered do still have relevance to current automation technology. However, the concept of a “monitor” is now more generally a user, who may be required to conduct some monitoring, but is also likely to be more actively in control in some elements
216
S. Sharples
of the task, and will work with the automation, along with other technologies and artefacts, as part of a joint cognitive system. Similarly, the original paper by Bainbridge focussed on the “process” whereas it is now more appropriate to phrase the statements in terms of the overall system. In all three cases, the concept of individual differences in preferences for a style of automation use is evident. This, coupled with the element of user choice over the use of automation, leads to the consideration of a new irony, that despite the aim of some automation systems to protect the user from its complexity, it is only by understanding this complexity that the user is able to form a mental model and take advantage of the full capability of the automation by working with it as part of a joint cognitive system. This requires a deliberate act on the part of the user to embrace the automation.
References Artman, H. & Garbis, C. (1998) Situation Awareness as Distributed Cognition. In Cognition and Co-operation. Proceedings of the 9th Conference of Cognitive Ergonomics. Limerick, Ireland. Bainbridge, L. (1983). Ironies of automation. In J. Rasmussen, K. Duncan & J. Leplat (Eds.), New Technology and Human Error (pp. 271–283). Chichester: Wiley. Balfe (in preparation) Automation in rail signalling. PhD thesis, University of Nottingham. Balfe, N., Wilson, J.R., Sharples, S. & Clarke, T. (2007). Analysis of current UK rail signalling systems. Paper presented at the Human Factors and Ergonomics Society European Chapter conference on Human Factors for Assistance and Automation, Braunschweig, Germany, October 2007. Balfe, N., Wilson, J.R., Sharples, S. & Clarke, T. (2008). Structured Observations of Automation Use. In P. Bust (Ed) Contemporary Ergonomics 2008. Burnett, G.E., and Lee, K. (2005) The effect of vehicle navigation systems on the formation of cognitive maps, in G. Underwood (Ed.) Traffic and Transport Psychology: Theory and Application. Elsevier. Hollnagel, E. (2001). Cognition as Control: A Pragmatic Approach to the Modeling of Joint Cognitive Systems. Special issue of IEEE Transactions on Systems, Man and Cybernetics – Part A: Systems and Humans. Lane, D.M., Napier, H.A., Peres, S.C. & Sandor, A. (2005) Hidden costs of graphical user interfaces: failure to make the transition from menus to icon toolbars to keyboard shortcuts. International Journal of Human-Computer Interaction. 18(2), 133–144. Nichols, S., Bristol, N. & Wilson, J.R. (2001) Workload assessment in Railway Control. In D. Harris (ed) Engineering Psychology & Cognitive Ergonomics. Artman & Garbis (1998). Nixon, J. Sharples, S. & Jackson, M. (2008) Less is more? Navigating with different types of information on a small-screen device. Paper accepted for presentation at the Human Factors and Ergonomics Society’s 52nd Annual Meeting. New York: September 22–26.
Automation and technology in 21st century work and life
217
Norman D.E. (1988) The Psychology of Everyday Things. New Your: Basic Books. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 30(3), 286–297. Rogers, Y. & Scaife, M. (1999) How can interactive multimedia facilitate learning? In Lee, J. (Ed.) Intelligence and Multimodality in Multimedia interfaces: Research and Applications. Menlo Park, CA: AAAI Press. Zimmer (2004) The Construction of Mental Maps Based on a Fragmentary View of Physical Maps. Journal of Educational Psychology 96(3), 603–610.
VIGILANCE AND HUMAN SUPERVISORY CONTROL – A POTTED HISTORY J.M. Noyes University of Bristol, UK The interaction of humans and machines in terms of task allocation and control issues is an important consideration in the design and operation of systems. With advances in technology and greater possibilities concerning ‘intelligent’ systems, the role played by the human is crucial to the success of the system. Too much human control, and the benefits of automation may not be realised; too little, and the human supervisor may be at a loss to deal with failures and malfunctions when they occur. This paper is the first of three papers being presented here on this topic. Its aim is to review the development of supervisory control since the 1940s in order to set the scene for the following two papers.
Clock watching A major milestone in the study of human supervisory control occurred in 1948, when Norman Mackworth devised the simple but very elegant clock test in order to evaluate vigilance in UK military personnel (Mackworth, 1948). Participants had to observe a clock-like device and look for double-step movements (the target) among single-step movements (the non-target). Typically, Mackworth found that there was a decline in detection rate with time on task and an increase in detection time. By carrying out a series of experiments, he demonstrated that individuals could improve their vigilance performance by having shorter working periods, taking breaks, being given feedback, and in the short term, by ingesting amphetamines (Mackworth, 1950). Mackworth’s work is seminal for various reasons. It draws attention to the problem of vigilance when humans are monitoring machine-based systems. The Second World War, unlike the First World War, had been fought mainly in the air, and this had necessitated equipment being designed and implemented at great speed without sufficient opportunity for testing and revision. As an example, radar operators had been placed on 8-hour shifts, and Mackworth had observed that a large number of targets were being missed after a relatively short period of time. Surprising though it may seem now, knowledge about human attention span and monitoring skills, especially when checking for low frequency targets, and the application of this knowledge to the workplace, was limited. Hence, Mackworth began to lay the way by studying human interactions with technology, and in particular, the allocation of functions issues which still test us today. 218
Vigilance and human supervisory control – a potted history
219
The ‘big issue’ in a world which has seen increasing use of technology over the last 60 years concerns human supervisory control. Too much human control and the benefits of automation are not realised; too little and the human operator is not ‘in the loop’ when failures and malfunctions occur. This is particularly evident on the civil flight deck where computer-based systems can very safely control the aircraft (and often with greater accuracy than the human pilot). However, when problems arise, systems will usually shut down one by one, before handing (ultimate) control back to the human. This person (or persons) then has to rectify a situation but with various gaps in their knowledge about what exactly has gone wrong. Thus, automation-induced dependence needs to be avoided. Can history help us? Automation is not a new problem. How over the last 60 years have we attempted to address this issue of human supervisory control? This paper will consider the historical development of advanced technology with particular reference to automation, and human and machine control of functions to see what we have learnt from the past.
1. Automate as much as possible In the immediate post-war years of the 1940s and ‘50s, the trend was to automate as much as possible. This was summed up by Bainbridge (1987, p. 271) who suggested that “the classic aim of automation is to replace human manual control”. The drive to automate may have been in part due to the novelty and newness of having systems and machinery, which could replace human activities. The obvious example is the development of computers around this time; for example, the replacement of the vacuum tube with the transistor heralded the second generation of computers, and hence, the creation of the transistor in the late 1940s as been cited as one of the greatest inventions of the 20th century (see, Herrick, 2003). Another factor driving the ‘automate as much as possible’ approach is likely to have been human error. Humans make errors, and generally, errors have negative connotations. One way to prevent human error is to keep people away from the system; ensuring that they do not have direct contact should mean they cannot make errors. In theory, this will work up to a point, since it is difficult, if not impossible, to remove human involvement completely from a system. Take a highly automated nuclear power plant. Humans may not be carrying out minute-to-minute monitoring activities, but they will be responsible for overall control, and stopping production should the need arise. In terms of human supervisory control, as a general rule, tasks which did not lend themselves to automation were allocated to humans. This subsequently became known as the ‘left-over approach’; that is, humans ended up with the tasks which the systems could not do. Not surprisingly, this approach had a number of drawbacks; the most obvious one being that the human was then left to do the tasks which were impossible for the automated system. Pragmatically, this does not make sense. It is likely the tasks which are difficult for the system are equally problematic for the human operator. As an example, take a commercial aircraft landing in adverse weather conditions. This may prove a hazardous exercise regardless of whether a
220
J.M. Noyes
manual or auto-pilot landing is attempted, and indeed, there will be some situations where a manual landing has to take place in preference to an auto-pilot landing, and of course, vice versa. In summary, the trend in the immediate post-war years was to automate as much as possible but this inevitably resulted in tasks, which could not be carried out by the system, being allocated to the human. This inevitably creates problems for the human in terms of supervisory control as they are likely to end up with the ‘hard’ tasks which are not possible for execution by the system.
2. Automate according to human and machine characteristics In the 1950s, the so-called ‘left-over approach’ was modified into one which took into account the attributes of humans and machines, and allocated the various functions according to the strengths of each faction. This was based on the work of Paul Fitts; he considered a number of functions relevant to the operation of a system in terms of how humans and machines would fare, and this became known as a Fitts’ List (Fitts, 1951). An example of some of the functions, which might be considered, is given in Table 1.
Table 1.
Example of a Fitts’ List approach (from Noyes, 2001).
Function
Humans
Machines
Speed
Can work fast for short bursts, limited reaction times, comparatively slow, for example, seconds
Accuracy
Can be extremely accurate, but again difficult to sustain over periods of time Soon tire, subject to learning, fatigue and boredom. Poor, especially when required to sustain attention over long periods
Can work fast for long periods of time, much faster reaction times, for example, up to the speed of light Can maintain high standards of accuracy over long periods of time Excellent at repeating actions in a consistent manner Not an issue – if programmed to monitor, will continue to do this ad infinitum Poor (unable, for example, to carry out simple tasks such as edge detection) Completely reliable in that all items stored in memory can be quickly accessed, limited (but extremely large) capacity, formal structure Rule-based approach, good at deductive reasoning, poor in new situations Limited
Repetitive actions Attention span
Perceptual skills
Excellent especially at pattern detection and interpretation
Memory
Selective, impressive in that unprompted instant retrieval of items is possible, not always reliable, versatile and innovative
Decision making
Tend to use heuristics (‘rules of thumb’), good at inductive reasoning when need to draw on experience Excellent
Intelligence
Vigilance and human supervisory control – a potted history
Table 2.
221
Some shortcomings of the ‘list’ approach to allocation of function (from Noyes, 2001).
1. Some functions have to be performed by humans because machines cannot do them, and vice versa. 2. There will be some functions where allocation is not obvious, that is, they might be carried out by humans or machines or both. 3. It implies that tasks which machines do well need to be assigned to them leaving the remainder for the humans. 4. Direct comparison of attributes suggests that machines and people have opposite capacities. Obviously, this is not the case. 5. Some functions must always be performed by the human operator, for example, abort control. 6. Some functions may be allocated to the humans in order to make their jobs more interesting. 7. Technology changes faster than the Lists. 8. It does not cover all the reasons why the human may be allocated a ‘machine’ function. 9. Being skilled at a task is only one facet/dimension. 10. Humans have a need to work and feel valued. 11. They are also unique in that they have an emotional component. 12. Humans may have a preference to interact with other humans rather than machines.
This approach is also often referred to as a ‘compensatory principle’ (see, Fitts, 1951); this presumably derives from the fact that human and machine characteristics will compensate for each other. For example, humans are good at deductive reasoning whereas machines (computers) are better at inductive reasoning, so in a well-designed system, one aspect should compensate for the other to ensure the optimum type of reasoning is taking place. Later, these types of comparisons becoming known as MABA-MABA (“Men [sic] Are Best At – Machines Are Best At”) lists (see, Berry & Houston, 1993). The relatively simple approach to allocation of function manifested in the use of these types of lists has persisted over many decades. And as noted by Sheridan (2000, p. 203), they have been referred to as “a kind of gospel” for allocating functions to humans and machines. However, it is unlikely that Fitts and others intended them to be viewed in this way or indeed even for the list to be used for the allocation of tasks to humans and machines. Further, it is implicit in Fitts’ and MABA-MABA lists that humans and machines need to be compared in some way and this may not be the case. These are a few of the many shortcomings associated with the list approach; others are given in Table 2. In summary, allocation of function according to human and machine characteristics provides a relatively simple approach albeit with a number of shortcomings; however, it was an advance on the previous approach of automating as much as possible.
3. Automate flexibly taking into account overall system operation The 1970s led to a more integrative approach. It was recognised that human and machines/systems have their respective strengths and weaknesses, but they are both
222
J.M. Noyes
part of a common goal in terms of total system operation (see, Price, 1985). Thus, the emphasis shifted towards co-operation and the development of a symbiotic relationship between the two main players. This has become known as a complementary approach (Hollnagel, 1999). Engendered in this approach is flexibility; functions need to be allocated according to the goals and characteristics of the system at that time. On the flight deck, there may be occasions when the same task can be safely carried out by the human, and times when the systems with their ability to sample every number of milliseconds with great accuracy and then make appropriate adjustments are safer. Alternatively in an emergency, humans with their ability to reason inductively taking into account, for example, previous knowledge and understanding, may fare better. As the adage states, flight deck crew have an incentive to get home safely, which the computers lack. Further strives were also made in the 1990s concerning human error. As already mentioned, earlier systems were developed with an error-resistance approach; that is, humans make errors, so if at all possible, they need to be kept away from the machines. This was replaced with a view which accepts that human operators will make errors and this needs to be taken into account when designing the system and subsequent human-machine interactions. This became known as error-tolerance (Billings, 1997). Two simple examples might include the ‘undo’ key in word processing packages, and the ‘are you sure’ request when executing an operation which may have serious consequences (e.g. deletion of files). In summary, a more flexible approach to supervisory control came into existence in the 1970s and this probably still exists today. It is certainly evident in the approach taken to accommodating human error in the design of systems, which represents a U-turn in terms of the role of the human operator.
4. And in the 21st Century . . . The notion of flexibility persists. The human is still at the centre of activities even with highly automated systems. Take signalling on the rail network. The operator can choose to anticipate or override the automatic route setting (which they might due given their greater knowledge to the signalling system concerning train movements and track availability). Flexibility is also evident in the work on automation. For example, Kaber and Endsley (2004) developed a taxonomy comprising 10 levels of automation ranging from manual control to full automation for four different human/computer tasks (monitoring, generating, selecting, implementing). Moreover, they also suggested that other factors such as situation awareness, workload, and trust/confidence in the system are relevant when considering human supervisory control issues (see, Parasuraman, Sheridan, & Wickens, 2008, for a recent paper on these very three factors). If we take the first factor, situation awareness, this is a relatively new term developed in the 1990s for what is an old concept which was recognised during the Second World War (Watts, 2004). Although there are numerous definitions for
Vigilance and human supervisory control – a potted history
223
situation awareness, if taken at face value, there are obvious implications in terms of its benefits in supervisory control. Like situation awareness, workload (both cognitive and physical) and its measurement have a long history; however, it was not until the 1980s that some of the self-report measures for subjective workload gained maturity. For example, the development of the Modified Cooper-Harper scale (MCH; Wierville & Casali, 1983), the Overall Workload (OW; Vidulich & Tsang, 1987), the NASA-TLX (Task Load Index; Hart & Staveland, 1988), the Subjective Workload Assessment Technique (SWAT; Reid & Nygren, 1988). Trust and confidence in the system, that is, believing in the accuracy of the information being displayed, is also an important factor. As a general rule, operators tend to make the assumption that the information is correct; that is, they trust the automation. At a pragmatic level, this makes sense. In an aircraft at 35,000 feet, it is not useful to begin questioning the automation. However, studies by Muir and Moray (Muir, 1994; Muir & Moray, 1996) have shown some interesting findings with regard to operators’ trust; for example, they showed that there was an inverse relationship between trust and monitoring of the automation. They went on to suggest that “operators’ subjective ratings of trust and the properties of the automation which determine their trust, can be used to predict and optimize the dynamic allocation of functions in automated systems” (Muir & Moray, p. 429). As highlighted by the Parasuraman et al. (2008) these three factors are more recent developments and for consideration in supervisory control. A further addition, which has not attracted much interest from designers in recent times, is user expectations (see, Noyes, 2006). Expectations (i.e. how operators expect a system to perform) are known to be an important aspect of human-machine interaction. Not taking into account, people’s expectations can lead to a mismatch with their perceptions; this can lead to frustration and irritations as well as errors when humans interact with the system. Noyes suggested that past experience and knowledge facilitate development of expectations and these are important determinants in our attitudes towards technology. She described a questionnaire survey of the expectations and perceptions of automated warning systems by British Airways flight deck crew. It was found that the automation was more favoured by crew who had experience of using warning systems with a greater degree of automation. Conversely, those without this experience had more negative expectations. Thus, lack of exposure to a system may lead to more negativity. As a final comment, it is suggested that teamworking needs also to be considered. Traditionally, experimental research has focused on performance of the individual, especially in the laboratory setting, and in fact, there is scant mention of teams as opposed to individuals when considering the history of supervisory control. It would appear that this needs to be addressed more fully in future studies.
What can we learn from history? Over the decades, different approaches to supervisory control have been taken with varying degrees of involving/excluding the human. However, there is no straightforward or obvious answer in terms of how best to allocate function between the
224
J.M. Noyes
human and the system. In terms of system operation and human involvement, the general move has been towards flexibility and a symbiotic approach whereby humans and machines work in conjunction with each other. Certainly, history seems to have taught us that you cannot take the human ‘out of the loop’. Ultimately, all machine-based activities are controlled by humans at some stage (from the initial ideas through conceptual design and development to operation, maintenance and decommissioning). It may be that this situation will change in the future, but for obvious reasons, it is probably unlikely. A starting point, therefore, seems to be to begin with the human and ensure the human operators remain at the centre of machine-based activities, and throughout the life cycle. Ignoring the human is not an option.
References Bainbridge, L. 1987, Ironies of automation. In J. Rasmussen, K. Duncan & J. Leplat (eds.) New Technology and Human Error, (John Wiley & Sons Ltd., Chichester, England) Berry, L.M. and Houston, J.P. 1993, Psychology at Work: An Introduction to Industrial and Organisational Psychology, (Brown & Benchmark, Madison, WI) Billings, C.E. 1997, Aviation Automation: The Search for a Human-centred Approach, (LEA, Mahwah, NJ) Fitts, P.M. 1951, Engineering psychology and equipment design. In S.S. Stevens (ed.) Handbook of Experimental Psychology, (Wiley, New York) Hart, S.G. and Staveland, L.E. 1988, Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P.A. Hancock and N. Meshkati (eds.) Human Mental Workload, (Elsevier, New York), 5–39 Herrick, D.F. 2003, Media Management in the Age of Giants: Business Dynamics of Journalism, (Iowa State Press/Blackwell, Ames, IA) Hollnagel, E. 1999, Control versus dependence: Striking the balance in function allocation, CSERIAC Gateway, IX(4), 12–13 Kaber, D.B. and Endsley, M.R. 2004, The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task, Theoretical Issues in Ergonomics Science, 5, 113–153 Mackworth, N.H. 1948, The breakdown of vigilance during prolonged visual search, Quarterly Journal of Experimental Psychology, 1, 5–61 Mackworth, N.H. 1950, Researches of the measurement of human performance. (Medical Research Council Special Report No. 268), (HMSO, London) [Reprinted H.W. Sinako (ed.) Selected papers on Human Factors in the Design and Use of Control Systems, 1961, (Dover, New York), 174–331 Muir, B.M. 1994, Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems, Ergonomics, 37(11), 1905–1922 Muir, B.M. and Moray, N. 1996, Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation Ergonomics, 39(3), 429–460 Noyes, J.M. 2001, Designing for Humans, (Psychology Press, Chichester, England)
Vigilance and human supervisory control – a potted history
225
Noyes, J.M. 2006, Expectations and their forgotten role in HCI. In C. Ghaoui (ed.) Encyclopedia of Human Computer Interaction, (Idea Group Inc., Hershey, PA) Parasuraman, R., Sheridan, T.B. and Wickens, C.D. 2008, Situation awareness, mental workload, and trust in automation: Viable empirically supported cognitive engineering constructs, Journal of Cognitive Engineering and Decision Making, 2(2), 140–160 Price, H.E. 1985, The allocation of function in systems, Human Factors, 27(1), 33–45 Reid, G.B. and Nygren, T.E. 1988, The Subjective Workload Assessment Technique: A scaling procedure for measuring mental workload. In P.A. Hancock and N. Meshkati (eds.) Human Mental Workload, (Elsevier, New York), 185–218 Sheridan, T.B. 2000, Function allocation: algorithm, alchemy or apostasy? International Journal of Human-Computer Studies, 52, 203–216 Watts, B.D. 2004, “Situation awareness” in air-to-air combat and friction. Chapter 9 in Clausewitzian Friction and Future War, McNair Paper no. 68 (revised edition; originally published in 1996 as McNair Paper no. 52). (Institute of National Strategic Studies, National Defense University) Wierville, W.W. and Casali, J.G. 1983, A validated rating scale for global mental workload measurement application. In Proceedings of the Human Factors Society 27th Annual Meeting, Santa Monica, CA (129–133) Vidulich, M.A. and Tsang, P.S. 1987, Absolute magnitude estimation and relativejudgment approaches to subjective workload assessment. In Proceedings of the Human Factors Society 31 st Annual Meeting, Santa Monica, CA (1057–1061)
HUMAN ERROR/ACCIDENTS
UNPACKING THE BLUNT END – CHANGING OUR VIEW OF ORGANIZATIONAL ACCIDENTS B.M. Sherwood Jones Process Contracting Limited Learning from incidents is a fundamentally different form of enquiry to investigations for the purposes of blame or compensation. However, the models we use to describe our understanding of incidents have still to take this into account. For example, ‘accident trajectory’ is a concept that builds in hindsight bias from the outset. Treating ‘the organization’ as a single entity does not reflect the complex network of organizational agents involved in most incidents. The lessons from the study of safety-related organizations (e.g. they run broken most of the time) have still to be absorbed. The concepts of ‘cause’ and ‘root cause’ still persist, even though ‘emergence’ is well established as a property of socio-technical systems. We are now at a stage where a fresh approach using systems thinking is both possible and useful. An example of such an approach, termed the Swiss Chateau Model, is proposed.
Introduction The book that really started the study of organizational accidents was Turner (1978). The three models presented ten to fifteen years ago by Moray (1994), Rasmussen (1997) and Reason (1997) have shaped recent thinking (the existence of Reason (1990) is acknowledged, but Reason (1997) was the influential publication). The lack of attention given to Moray (1994) and Rasmussen (1997) represents a missed opportunity to take a systems approach, but that seems to be the situation. The dominant model has been the Reason (1997) Swiss Cheese Model, shown in Figure 1 below (situated outside Gruyere). It has been very widely adopted and
Figure 1.
Swiss Cheese Model. 229
230
B.M. Sherwood Jones
undoubtedly been a force for good. Recently the Swiss Cheese Model has been the subject of some discussion (e.g. Perneger, 2005), with a significant contribution from Reason himself (Reason et al., 2006). Much of the discussion has been about the balance between individual and organizational (active and passive) failures. The evidence on this (Johnson, 2006a) seems to be that the “80% human error” figure continues to be unhelpful and a greater emphasis on organizational errors is still warranted.
Models of organizational accidents The discussion of models of organizational accidents is hampered by two issues. Firstly, the discussion is still in terms of ‘cause’. Organizational accidents are emergent properties of complex systems (Weinberg, 1975). As such, discussions about ‘cause’ are conceptually inappropriate. Discussion of the ‘influence’ of ‘patterns’ would be philosophically correct, and practically helpful (Snowden and Boone, 2007). Secondly, the purpose of current accident models is unclear; possibly they are to provide some sort of understanding or reconstruction. However, their structure induces massive hindsight bias, particularly the Swiss Cheese Model concept of an ‘accident trajectory’. Hindsight bias is the biggest problem in accident investigation (Dekker, 2002), and so current models are limited even in preventing a repeat accident. It could be argued that Leveson (2004) is an exception to the neglect of systems approaches. Her approach claims roots in socio-technical systems, and acknowledges the need to recognise emergence and the limitations of event chain models. However, the STAMP method emerges as a ‘new model of accident causation’ (Laracy, 2007 emphasis added), based on hierarchical control loops, rather than examining the emergent properties of a network of identified purposeful activity systems. Despite recognition of the nature of complexity (by citing Weinberg, 1975), the method uses ordered-systems thinking (Kurtz and Snowden, 2003) and is inapplicable to the un-ordered space occupied by networks of interacting agents. A full practical understanding of complexity (Snowden and Boone, 2007) has not been used. Despite acknowledging that change is the only constant, the approach, like traditional economics, makes the inappropriate assumption of a state of equilibrium (Beinhocker, 2007).
The ‘blunt end’ The emphasis in the organizational accident literature has been on the ‘sharp end’ despite recognition that the ‘blunt end’ affects safety through the effects of ‘the’ wider system on resources and constraints available to the sharp end (Cooke and Woods, 1994). Treating all the stakeholders from middle management to government regulators as a single system labelled ‘the blunt end’ is not helpful to our understanding of organizational accidents. Incident analysis needs to unpack the blunt end into more specific systems. The approach to incident analysis proposed by Dekker (2002) has yet to be adopted by formal investigatory bodies. Its adoption would enable analysis practice
Unpacking the blunt end – Changing our view of organizational accidents
231
to take a large step forward in both conducting analysis and in making recommendations. The singular, surprising, omission from his book (op. cit.) is extending the analysis from its exclusive focus on the sharp end to gaining an understanding of the evolving mindset in design offices or regulatory drafting groups. There has been little investigation of the role of policy and regulation. Johnson (2006b) is a notable exception, Macrae (2007) is a rare example of understanding the evolving mindset, and Lassagne (2005) is extremely rare in examining regulatory capture (though without using the term). It has occasionally happened that the regulator has been the object of investigation and prosecution (Booker and North, 2007). A suitable structure for the systems in the blunt end needs to be adopted. At a high level, this could be a model such as the Business Excellence model (EFQM, 1998), or a system engineering life cycle (ISO/IEC, 2008), supported by more localised models of good practice. The starting place needs to be action, rather than the accident. Such a change also removes the negativity of Generic Failure Types (Reason, 1997).
Requirements for a useful model From a practical point of view, it would be logical to try to prevent the next accident rather than the last one. The difference between the two tasks has a major impact on the model of organizational accidents required: the model would need to support a wider range of analysis and improvement actions. Features of a model that might help people to learn from incidents and prevent the next one include: 1. Recognition that the operator is the last line of defence, rather than the first. It appears to be an unintentional affordance of the Swiss Cheese Model that gives support to the idea that the crew should be the first line of defence. Pilots are always the first to arrive at the scene of the accident, but it is the responsibility of the other stakeholders to ensure that pilots are placed in manageable circumstances. The role of crew as a barrier should be seen as equivalent and downstream to that of say, a regulator. Human errors need to be seen as equivalent, rather than fundamentally different (cf. ‘active’ and ‘passive’). 2. Recognition that ‘the organization’ is not a single homogeneous entity, or a hierarchy, but rather a network of interacting agents. 3. A clear link to a framework for action and improvement. 4. The ability to reconstruct the evolving mindset (Dekker, 2002), rather than plot an accident trajectory. 5. Recognition that complex systems are not inherently safe, with a few holes in generally complete layered defences, but need constant effort to keep making them safe (Dekker, 2002, Cooke and Woods, 1994).
Proposed model – Swiss Chateau Model The proposed model has been situated in Swiss cheese country to represent evolution rather than revolution, and is the Swiss Chateau Model, illustrated in Figure 2
232
B.M. Sherwood Jones
Figure 2.
Figure 3.
Swiss Chateau Model.
Swiss Chateau Model elements.
below. From Reason et al. (2006) it appears that visual affordances of a model are important, and that a considerable part of the use and abuse of the Swiss Cheese Model has been from its affordances rather than the supporting theory. The Swiss Chateau Model emphasises that the outer defences to safe and effective operation are provided by the ‘enabling systems’ (ISO/IEC, 2008) i.e. the various organizational stakeholders. The inner defence is provided by the sharp end operators – the crew. There are no longer layered defences, and the single outer defence ‘runs broken’. Six enabling systems are identified, shown in Figure 3. Three systems are near the crew in temporal terms and generally operating on short timescales. Local Management is the immediate management and supervision. Recovery includes both containment and recovery. An important feature of a High Reliability Organization is the ability to bring in extra resources rapidly under circumstances of high demand. Learning From Experience is the system for learning from near-misses, surprise successes and incidents. In the case of a ship running aground, Crew is the on-watch
Unpacking the blunt end – Changing our view of organizational accidents
Figure 4.
233
Longer term enabling systems.
bridge team. Local Management is the Master and his direct reports. Recovery includes the company response team, the relevant Coast Guard and rescue services, support from Class and passengers with mobile phones. Learning From Experience includes debriefings, company and other near-miss reporting systems. Even under normal conditions, Local Management has direct dealings with all enabling systems. Working arrangements do not reflect any abstract control hierarchy, and it is not clear why models of accidents should. The longer term enabling systems shown in Figure 4 below are as follows. Operations comprises the management of the enterprise that runs the system of interest e.g. a shipping company for a system of interest that is a ship. Design and Build embraces the supply chain for the system of interest. For an aircraft it might be Boeing and all its suppliers. Regulation comprises the regulatory system (which will be more than just a single regulatory body). Analysis of enabling systems, including the longer term systems, examines the evolving mindset e.g. why it made sense in a design office to remove an alarm channel and replace it with a different one. The frame of reference is not accident causation, but oriented to action e.g. shortfalls from good practice. If learning from the incident is to take place, then the lessons need to be absorbed into the body of narrative fragments used by the enabling systems, rather than residing in a report on a shelf. The ‘cycle of error’ (Cooke and Woods, 1994) can be avoided by choosing corrective actions that are appropriate to the Cynefin domain (Snowden and Boone, 2007).
Implications of a new model The first implication is a change in approach; many new stakeholders are put in the front line. This may help to shift the perceived importance of organizational and individual errors. The use of enabling systems as a structure assists analysis. It enables the evolving mindset of each of the stakeholders to be determined. This can be very difficult
234
B.M. Sherwood Jones
in conventional incident reports. The structure also aids improvement action by addressing the full range of stakeholders directly. If incident analysis is to help to prevent the next accident rather than the last one, then the capability of each enabling system needs to be assessed (ISO/IEC, 2004) against good practice, rather than an evaluation of outcomes against some assignment of cause. In practical terms, there are likely to be a good many fairly immediate implications for enabling systems; a pre-emptive self-assessment might be a good start. Incident analysis is in a transitional stage; there is an awareness that the sharp end is just the tip of an iceberg, but the field has not really taken a remit to go below the surface to the extent that would deliver real improvements in safety. Safetycritical organizations appear to be taking continuous improvement more seriously, schemes for near-miss reporting are growing, and a connection between ‘learning from incidents’ and more general organizational learning may well be accepted. A new framework is needed quickly if an opportunity is not to be missed (again). The need for a more usable incident analysis paradigm is pressing. This will require a move away from bulky incident reports and databases of categorised incidents. Our understanding of how to manage in a complex context has advanced considerably (Snowden and Boone, 2007), and social networking tools to apply that understanding are becoming available.
Acknowledgements The author would like to thank Lloyd’s Register for the opportunity to explore and act on many of these ideas, and two anonymous reviewers for very helpful comments. All errors are the author’s own and the ideas expressed are personal. Some of the material has been published previously under a Creative Commons licence.
References Beinhocker, E.D. 2007, The origin of wealth. Evolution, complexity and the radical remaking of economics (Random House Business Books) Booker, C. and North, R. 2007, Scared to Death: From BSE to Global Warming: Why Scares are Costing Us the Earth (Continuum International) Cook, R. and Woods, D.D. 1994, Operating at the Sharp End: The Complexity of Human Error in Bogner, M.S. (ed.) Human Error in Medicine (Lawrence Erlbaum) Dekker, S. 2002, The Field Guide to Human Error Investigations (Ashgate) EFQM 1988, Excellence Model (European Foundation for Quality Management) ISO/IEC 15288:2008 System engineering - system lifecycle processes (International Standards Organization) ISO/IEC 15504-4:2004 Information technology – Process assessment – Part 4: Guidance on use for process improvement and process capability determination (International Standards Organization)
Unpacking the blunt end – Changing our view of organizational accidents
235
Johnson, C.W. 2006a, On the Over-Emphasis of Human ‘Error’As A Cause of AviationAccidents: ‘Systemic Failures’and‘Human Error’in US NTSB and Canadian TSBAviation Reports 1996–2003 (http://en.scientificcommons.org/c_w_johnson) Johnson, C.W. 2006b, Establishing Public Policy as a Primary Cause of Engineering Failure: Did Market Deregulation Lead to the North American ‘Blackout’, August 14th 2003? in Balducelli and Bologna (eds.) Proceedings of the ENEA International Workshop on Complex Networks and Infrastructure Protection (ENEA) Kurtz, C. F. and Snowden, D. J. 2003, The new dynamics of strategy: Sense-making in a complex and complicated world, IBM Systems Journal, 42, 3 Laracy, J.R. 2007, Addressing System Boundary Issues in Complex Socio-Technical Systems (CSER 2007 Proceedings) Lassagne, M. 2005, A Tale of Two Laws : Private and Public Regulatory Organizations in the Maritime Shipping Industry XXIe Colloque EGOS (European Group for Organization Studies) Leveson, N. 2004, A New Accident Model for Engineering Safer Systems, Safety Science, 42, 4, 237–270 Macrae, C. 2007, Interrogating the unknown; risk analysis and sensemaking in airline safety oversight. Discussion Paper 43 (LSE Centre for analysis of risk and regulation) Moray, N. 1994, Error reduction as a systems problem in Bogner, M.S. (ed.) Human Error in Medicine (Lawrence Erlbaum) 255–310 Perneger, T.V. 2005, The Swiss cheese model of safety incidents: are there holes in the metaphor? (http://de.scientificcommons.org/8444947) Rasmussen, J. 1997, Risk Management in a Dynamic Society: A Modelling Problem. Safety Science 27, 2/3, 183–213 Reason, J. 1990, The contribution of latent human failures to the breakdown of complex systems (Philosophical Transactions of the Royal Society) series B. 327: 475–484 Reason, J. 1997, Managing the Risks of Organizational Accidents (Ashgate) Reason, J., Hollnagel E, Paries, J. 2006, Revisiting the “Swiss Cheese” model of accidents EEC Note No. 13/06 (Eurocontrol Experimental Centre) Snowden, D.J. and Boone, M.E. 2007, A leader’s framework for decision making Harvard Business Review Turner, B. 1978, Man-made disasters (Butterworth-Heinemann) Weinberg, G.M. 1975, An Introduction to General Systems Thinking (Dorset House)
LESSONS FROM THE PAST TO GIVE TO THE FUTURE: LEARNING FROM INCIDENTS S. Evans Rail Safety and Standards Board, UK The Rail Safety and Standards Board (RSSB) Human Factors (HF) team have developed a systems approach to learning from incidents. This approach comprises three work streams which together aim to increase the UK rail industry’s capability to learn the HF lessons. Firstly, an overview of an ongoing HF analysis of formal inquiries and incident investigation reports will be given. This will describe a classification system and database developed by the RSSB HF team to facilitate the identification of the multi-causal nature of incidents. This work was initiated to highlight the benefits of identifying the type and frequency of different error types and performance shaping factors (PSFs) in order to inform industry of (i) the human contribution to risk and, (ii) identify future HF priorities for the industry. Secondly, an overview of a supporting HF awareness training course to aid the incident investigation process will be provided. Finally, HF integration with RSSB’s strategy to improve the validity and reliability of safety-related information recorded in the industry’s Safety Management Information System (SMIS) will be highlighted.
Introduction Understanding the human contribution to risk There is a requirement to understand the human contribution to risk in order to increase the UK rail industry’s capability to learn human factor (HF) lessons from incidents. This paper describes three inter-related HF work streams which together feed into RSSB’s overarching strategy, referred to as ‘Learning from Operational Experience’.
Background In 2005, RSSB carried out a review of the industry’s Safety Management Information System (SMIS) (RSSB, 2005). SMIS is an industry-wide database used to record safety information on past incidents. The information is used to analyse risks associated with incidents at a national level. In the railway context, incident types include signals passed at danger, train derailments, collisions, train maintenance, broken rail etc. (RSSB, 2006). Although SMIS is thought to be a good source of risk data, it does not currently provide causal incident information which is usable from a HF perspective. Specifically, SMIS does not provide sufficient information on the type and frequency 236
Lessons from the past to give to the future
237
of different error types, thereby increasing the difficulty of identifying the human contribution to risk (RSSB, 2007).
HF learning from incidents – a systems approach Feeding into RSSB’s ‘Learning from Operational Experience’ strategy are three HF work streams which aim to increase the UK rail industry’s capability to learn HF lessons from incidents. This includes: 1. HF review of railway incident investigation reports 2. HF Awareness Training Course to aid incident investigations 3. HF contribution to SMIS Vision to improve the validity and reliability of HF incident causation information
Work stream 1 HF review of railway incident investigation reports The RSSB HF team have developed an evidence-based approach to reviewing the multi-causal nature of railway incidents (RSSB, 2007).This has involved the HF analysis of investigation reports. The aim was to highlight the benefits of identifying the type and frequency of different error types in order to inform the rail industry of the human contribution to risk and identify HF priorities, an approach supported by Network Rail and the Office of Rail Regulation.
Methodology The HF review of railway incidents included the development of a classification system and database in order to analyse the causal and contributory factors of incidents across the UK railway industry. A six-stage process was implemented: 1. A review was carried out to evaluate a number of existing incident analysis approaches. The classification system used in this study was based on the error classification model used in TraceR (Shorrock, 2006) and review of other classification approaches (e.g. Human Engineering, 2004). Given that the original error classification model used in TraceR was developed in the aviation domain, some aspects of the model did not translate directly to rail incidents. Adaptations were undertaken to the approach by the University of Nottingham (RSSB, 2004). Based on the information in the railway incidents analysed and inputs from railway operational specialists, the TraceR classification system has been further developed to reflect the nature of railway incidents. Further details are provided in the final report (RSSB, 2007). The aim of the review was to ensure the classification of error method did not omit any key error types or performance shaping factor. 2. A Microsoft Access database was developed to facilitate data classification and entry. This database has been modified during the project to reflect feedback on usability and coverage of contributing incident factors.
238
S. Evans
3. HF specialists within RSSB were trained on the analysis technique, classification of incident reports and use of the database. This was a key step to ensure consistent classification across incidents. 4. Monthly review of incident reports (approximately 20 reports per month). 5. Investigation reports were analysed and entered into the database by the RSSB HF team. Inter-rater reliability studies have been carried out in the development of TraceR (Shorrock, 2006). To ensure internal consistency, each report was classified by a minimum of two HF specialists. In the future, data already classified in the database will be verified by a formal inter-rater reliability study. 6. Ongoing review and refinement of the entire database.
Critical elements of the approach Incident analyses to investigate human performance issues are based on key concepts contained in theories of accident causation. The first is that incidents can be complex and be contributed to by a number of distinct factors, termed ‘incident’ factors. Secondly, incident factors can contribute to an event at the individual, job and/or organisational levels (Reason, 1999). A separate TraceR classification was used to categorise performance shaping factors (PSFs). Key categories include personal issues (e.g. stress, fatigue, and motivation), team factors (e.g. relationship with supervisor), workplace design (e.g. workload) and training issues (e.g. skills fade). Understanding the context of the error is also important. Therefore two elements to capture the context are included in the classification: (1) job role and personal characteristics of the person undertaking the task (e.g. experience) and (2) the task in which the error arose (e.g. driver observation of a signal). Incident reports can also reveal incident factors that relate to (1) the event itself, (2) post event factors which occur after the event but which potentially increase risk (such as incorrect train movement following a SPAD), and (3) incident investigation factors, which identify how the process of incident investigation could be improved (RSSB, 2007).
Identifying the human contribution to incidents Over six months of classification effort, 116 incident reports were reviewed and the relevant data entered into the database. The majority of report types analysed were formal inquiries and investigations (95%), the remaining being Rail Accident Investigation Branch reports and local investigations. An incident report may contain more than one type of event (for example a SPAD followed by a derailment). The most frequently occurring type of event was ‘SPAD Driver and Signal’. However, the high frequency of SPAD investigations is likely to reflect an industry bias towards investigating SPADs, rather than their relative frequency when compared to all incidents (RSSB, 2007). Frequencies for events across the industry can be more accurately derived from the Safety Risk Model (SRM) and other safety intelligence data. The SRM is a comprehensive mathematical representation of hazardous events that could lead directly to injury
Lessons from the past to give to the future
239
or fatality on the railway. It was developed using modelling techniques informed by a combination of incident data and expert judgement. The use of expert judgement reduces the problems that can arise with calculating risk estimates purely on the basis of data, which may be insufficiently representative of the underlying level of risk. Most of the data used to populate the SRM comes from SMIS. The method adopted in this study made a deliberate effort to look more widely than SPAD incidents by biasing the collection of events towards other event categories. 38% of the events analysed are from these other categories (e.g. derailments, collision, control of train movements, track worker protection from train movements). In the future, the event selection criteria could be further developed to match industry information requirements. Out of the 116 incident reports reviewed, 121 events were analysed (an incident report may contain more than one event). Events (e.g. derailments, SPADs, collisions) can be complex and can be contributed to by a number of factors, known as incident factors. Out of the 121 events analysed, 910 incident factors were identified. This represents an average of eight incident factors per event, illustrating the multi-causal nature of incidents. Of these 910 incident factors, 77% were errors and 23% were PSFs. This is interesting as it provides an indication of different areas that should be monitored and be placed under greater management control. The data can be analysed in a number of ways by cutting a ‘cross section’ according to the factors that are of interest. For example, it is possible to look at how incident factors relate to events. 82% of incident factors contributed to the event itself – these factors are key as they define why an event occurred. Interestingly, there was a relatively high frequency of post event factors (13%), which indicates that consideration must also be given to factors which arise after an event (e.g. correctly reporting a SPAD). 5% of incident factors related to the actual investigation process. To provide an overview of errors, the broader error types are summarised in figure 1. Person errors are related to a specific person in a defined job role and this category is used if an error can be attributed to a specific individual. Design errors cover both equipment design (e.g. train cab interface) and the wider system design. Management errors include safety management systems such as driver management, signaller management, Rule Book and Railway Group standards (RSBB, 2006). Diagram one illustrates that person errors dominate but there is a sufficient number of management and design errors to allow analysis of the different types of errors which contribute to incidents. The bias to human errors may reflect a bias in the investigations to consider the errors by front line staff rather than investigate the management systems which support them. Further review of the investigation process would be required to ascertain if this is the case.
In the spotlight – performance shaping factors The database allows errors to be examined in closer detail, which is of course important, as errors are the factors which can be identified and rectified to prevent recurrence. However, performance shaping factors are also important because
240
S. Evans
Person error Management error Design error
Figure 1.
Error Types.
they increase the likelihood of incidents occurring. Logically, if PSFs can be removed or mitigated, then the likelihood of errors occurring could also be reduced (RSSB, 2007) Table 1 provides an overview of the frequency of PSFs by category. These categories are defined as part of the TraceR approach. The dominant category is workplace design (50%) which covers the design of workplace elements which affect human performance. Examples of workplace design issues include conspicuity of alarms and warning devices, train cab layout, trust in equipment, workload and task complexity. Personal factors (25%) are also significant human performance shaping factors and include issues such as alertness, emotional or occupational stress, fatigue, job satisfaction and previous incident history. Environmental factors include issues such as lighting, noise and distraction, people within the train cab apart from the driver, and temperature. Training, knowledge and experience capture issues relating to the availability of training, familiarity with the task (such as route knowledge), and supervision. Social and team factors include team morale, and team relations. Communications issues include accent/dialect, company specific rail jargon and communication equipment and reliability. Finally, procedural issues that influence human performance include procedure complexity, procedure accuracy and compatibility with other processes or documentation. The database can narrow its focus to very low level detail, so that the category of interest can be studied in increasing detail. For example, if workplace design is of interest, the database can supply information on sub-categories so we can see which factors influence human performance in varying degrees. Time pressure is identified as being the largest contributing factor of workplace design, which takes into account 18% of all performance shaping factors, followed by distraction (10%), routine signal sequence (6%) and workload (2%). If we want to look at driver distraction in more detail to ascertain the lower level contributory factors, the data tells us that distraction as a PSF can be caused by train faults (33%), track workers (17%), distractions external to the cab (8%), low workload (8%) and others. PSFs can also be investigated by job role. For signallers, the two most frequently
Lessons from the past to give to the future
Table 1.
241
Overview of PSFs by category.
Performance Shaping Factor
Frequency
Percentage1
Workplace design Personal factors Environment Training, knowledge or experience Social and team factors Communications Procedures / documentation Total
106 53 19 17 10 4 1 210
50% 25% 9% 8% 5% 2% 0% 100%
1
Percentages rounded to nearest whole number
occurring issues are high workload and lack of familiarity. High workload is a factor which is an inherent part of the signalling task at certain times. Lack of familiarity is a factor which should be monitored, particularly as management errors in signaller competence were also identified as an area of concern.
Summary The HF review of incident investigation reports provides a powerful tool that has the capability of providing a comprehensive means of summarising rail incident factors for HF assessment. Usable outputs are being produced enabling incident factors to be built up across the UK rail industry. It is proposed that the database be used as an active tool to explore industry-specific issues and be provided for wider use throughout the industry to maximise its potential.
Work stream 2 HF awareness training course for incident investigators At the request of industry stakeholders, RSSB HF has developed a training course to raise awareness and promote understanding of HF causation (RSSB, 2008b). This training course has been developed as a ‘resource kit’, containing a number of different modules and tools that industry members can select and use as appropriate. The training course was developed for incident and accident investigators but would also be of interest to driver managers, trainers, and other personnel who have responsibilities for developing and maintaining staff competence. The content of the course is designed to introduce incident and accident investigators to HF. The course will be launched in February 2008 via ‘Train the Trainer’ sessions. These sessions will be delivered by the RSSB HF team to nominated trainers from Train and Freight Operating Companies. Company trainers will then deliver the training course to their colleagues in-house. The first training day provides a general introduction to HF and includes modules on human error and violations, understanding human performance, techniques for
242
S. Evans
investigation, and performance shaping factors at the individual level. The ‘Human Error’ module examines why errors and violations occur and provides tools to assist the incident investigator in the identification and classification of errors and violations. The ‘Understanding Human Performance’ module provides an overview of the human information processing system and explains how things can go wrong at the levels of perception, memory, decision making and situation awareness. Individual performance shaping factors are also discussed, such as fatigue, experience and training. Throughout the course, real-world examples are provided to illustrate how human error has contributed to incidents in the rail environment and other high-hazard industries, such as aviation, nuclear, and oil and gas. The second training day focuses on job and organisational factors that can contribute to incidents. Job factors, such as workload, equipment design and communication are discussed in detail, with an emphasis on how to identify risks that could lead to an incident. Organisation factors such as culture, leadership, systems and change are also discussed. The emphasis throughout the course is on practical application of the theory, and this is achieved through a range of exercises and group activities. Practical tutorials are included to provide trainees with hints and tips about classifying errors and violations, effective interviewing skills and techniques that can aid the investigation process. A toolkit is also provided to amalgamate the practical tutorials and is intended to aid the incident investigator whilst on-the-job. The toolkit includes useful questions for the incident investigator to ask while conducting a real-time investigation. These questions provide a checklist of prompts and reflect the current HF proposal for the industry-wide Safety Management Information System. As the training course has been primarily developed for incident investigators, it offers an excellent opportunity to map the process and terminology to the HF module in SMIS to ensure that both are consistent, which will maximise the accuracy of the data captured. This will be discussed further in the next section.
Work stream 3 HF contribution to SMIS Vision to improve the validity and reliability of accident causation information Improving the validity and reliability of industry-wide safety information The industry-wide database currently used to record safety information on past incidents is the Safety Management Information System (known as SMIS). RSSB, on behalf of the industry, is initiating a major programme to upgrade and enhance SMIS. This programme is known as the ‘SMIS Vision’ and is supported by a wide stakeholder group (RSSB, 2008c). One of the main drivers for the ‘SMIS Vision’ is to provide the industry with validated safety data that is accessible from one source. To be able to achieve this a number of improvements are needed to the current SMIS application and its related business processes. These include improving the flexibility and performance of SMIS, specifically for data entry inputters, industry agreement on the core and optional (company-specific) data requirement for collecting and recording
Lessons from the past to give to the future
243
safety-related data. It also includes making improvements to the quality of data being entered into SMIS. These objectives ensure that key data is validated and only available from one source. The HF team have been contributing to ‘SMIS Vision’ by proposing improvements to the underlying SMIS HF causation module. The objectives of the HF contribution to the SMIS Vision include: • Increased accuracy: to improve the reliability and validity of HF related incident information. • Increased usage: to increase the frequency of HF related incident information. • Easy identification: to make the database as easy as possible for incident investigators to identify HF causal factors. • Easy input and extraction: to make it as easy as possible for interested parties to identify trends and patterns related to HF from the incident information entered in SMIS. The links with RSSB’s HF Awareness training course for incident investigators encourages increased usage of SMIS and greater accuracy in the recording of error classification and performance shaping factors within it.
Conclusion RSSB are working with stakeholders to develop an industry-wide strategy known as the ‘Learning from Operational Experience’. This paper has discussed three work streams that the RSSB HF team have contributed to this strategy. This included the HF review of railway incident investigation reports, a bespoke training course to raise awareness of HF to aid the accident investigation process, and refinement of the HF module in SMIS. These three work streams are integrated to ensure a systems approach to learning from incidents. The process of incident report review and classification in the database have highlighted how improvements could be made to incident investigation processes from a HF perspective. The HF Awareness training course offers a number of techniques and methods to capitalise on the database results to train those responsible for investigations in the rail industry. This course has also provided an excellent opportunity to map the process and terminology of HF issues covered in the course with that used in SMIS to maximise the accuracy and validity of the data captured. This work will also enable changes in causal patterns to be monitored to shape the HF strategy across the industry. These combined approaches deliver a strategy by which the UK rail industry can better learn HF lessons from incidents.
References Rail Safety and Standards Board (2005). The classification of causes related to human factors within SMIS.
244
S. Evans
Rail Safety and Standards Board (2007). Human Factors Review of Railway Incidents: Phase 1 Report. RSSB research project T635. Human Engineering. (2004). SPAD Hazard Checklist. http://www.opsweb.com Rail Safety and Standards Board (2004). Rail-Specific HRA Tool for Driving Tasks. RSSB research project T270 http://wwww.rssb.co.uk/publications.asp Rail Safety and Standards Board. (2008a). Understanding human factors: a guide for the railway industry http://www.rssb.co. uk/expertise/human_factors/ human_factors_good_practices_guide.asp Rail Safety and Standards Board (2008b). Human Factors Awareness Training Course. RSSB research project T635. Rail Safety and Standards Board (May 2008c). Human Factors Strategy Document: Human Factors Contribution to the SMIS vision. RSSB research project T635. Reason, J. (1999). Human error. Cambridge University Press. Shorrock, S.T. (2006). Technique for the Retrospective and Predictive Analysis of Cognitive Error (TRACEr and TRACEr-lite) in International Encyclopaedia of Ergonomics and Human Factors Edited by W. Karwowski. CRC Press, Boca Raton, Florida, USA, 2nd Edition, Volume 3 pp. 3384–3389.
THE DEVELOPMENT OF HSL’S SAFETY CLIMATE TOOL – A REVISION OF THE HEALTH AND SAFETY CLIMATE SURVEY TOOL C. Sugden1 , M. Marshall2 , S. Binch1 & D. Bottomley3 1
Health and Safety Laboratory Health and Safety Executive 3 Merlin Analytical Solutions
2
The Health and Safety Laboratory (HSL) have revised and updated the Health and Safety Climate Survey Tool (CST) so that it is a more reliable and valid psychometric instrument, and as a result it is more useful to both public and private sector organisations. The revised Safety Climate Tool, SCT, contains 40 statements that map onto 8 factors that are more coherent and meaningful than the old factors. The revised tool now takes less time to complete, is written in clearer English and leading or ambiguous items have been removed. The revised tool has been subject to a period of consultation and a series of piloting exercises at various companies in the manufacturing, construction, and major hazards sectors. This paper will outline how the changes were made and give details of the tool’s successful application to date in industry. New software is under development for release in 2009.
Introduction Organisational climate is based on an aggregation of employees’ perceptions of their experiences within an organisation (Dawson et al., 2008). Safety climate, as a term was initially used by Zohar (1980) to describe attitudes towards safety, and was derived from earlier work on organisational climates. Typically safety climate is explored through questionnaires exploring attitudes and perceptions regarding safety; it is a statistical construction of perceptions held in an organisation regarding safety and is a way to summarise these (Rousseau, 1988). Cox and Flin (1998) in a review of academic papers on the issues stated that the terms safety culture and safety climate are used interchangeably to refer to similar concepts. Safety culture, as a phrase, was first used by the International Atomic Energy Authority to describe the issues at Chernobyl at the time of their major incident (IAEA, 1986). The interest in culture arose in response to a realisation that organisational structure (i.e. the roles and their relationships, rules and procedures) was limited in achieving an organisation’s health and safety goals. Hence the formal systems should prevent accidents occurring, for example by prioritising resource allocation, assessing training needs, adopting risk assessment methodologies and choosing tolerable risk criteria. HSE stated that the explicit and implicit goal of a 245
246
C. Sugden et al.
safety management system is the development of a positive safety culture (HSE, 1991). This reflects the understanding that safety management systems only work if individuals are motivated to comply and conform with the organisation’s systems, which is where the need to understand the impact of safety culture arose. HSC (1993) defined safety culture as: “the product of individual and group values, attitudes, perceptions, competencies, and patterns of behaviour that determine commitment to, and the style and proficiency of, an organisation’s health and safety management. Organisations with a positive safety culture are characterised by communications founded on mutual trust, by shared perceptions of the importance of safety and by confidence in the efficacy of preventive measures.”
HSE’s Health and Safety Climate Survey Tool The Health and Safety Climate Survey Tool (CST) was published by HSE in December 1997 (HSE, 1997); from then until 2007 over 800 copies of the tool were sold. The CST was designed to gauge perceptions of the management of health and safety, and associated issues recognised as important to prevent occupational accidents and incidences of ill-health. The supporting documentation stated that safety climate is a measurable component of safety culture, and identifies tangible issues that should help improve an organisation’s safety culture. Furthermore they highlight the importance of carrying out climate surveys to promote employee involvement and engagement. The CST was a computer-based product that enabled organisations to customise a safety climate questionnaire for use within their organisation. The software allowed organisations to analyse the results of their survey and produce graphs and reports on the basis of these findings. The software allows comparisons to be made across organisational hierarchies, and other groups of workers based on statement responses. Differences are often found between these groups and HSE (1997) stated that understanding these differences is central to making improvements to an organisation’s safety culture. Early in 2007, HSE withdrew the tool from the market, as the software was not compatible with later versions of Windows. However, the Baker Panel report into BP’s Texas City incident (The B.P. U.S. Refineries Independent Safety Review Panel, 2007) and their pronouncements concerning safety culture revived interest in the topic. This project was initiated to address some of the CST’s perceived shortcomings (e.g. long-winded, out of date, repetitive) and to address perceived market needs. The remainder of this paper discusses the updating process.
Objectives of the revision to the Health and Safety Climate Survey Tool The initial aims of this work were: • Reduce the size of the survey tool such that it could be completed within 10 minutes, but not reduce its power as an evaluative tool;
The development of HSL’s safety climate tool
247
• Derive a factor set that was more internally coherent and comprehensive that that contained within the original tool, building on the historic accumulation of data; • Use plainer English; • Produce upgraded software.
Methods The original CST (HSE, 1997) had 71 statements grouped into 10 factors: • • • • • • • • • •
Organisational commitment and communication; Line management commitment; Supervisor’s role; Personal role; Workmates influence; Competence; Risk-taking behaviour & some contributory influences; Some obstacles to safe behaviour; Permit-to-work systems; Reporting of accidents and near miss systems.
Each statement required a response on a 5-point scale, ranging from “strongly agree” to “strongly disagree”. When coded for use in analyses, some questions had their responses reflexed to ensure that they all worked in the “same direction” (that is, a shared concept between a pair of variables is reflected in a positive correlation). Some users of the CST participated in a benchmarking scheme, which generated a large database of industry responses. This dataset was made available to the HSL project team, and comprised of almost 40,000 questionnaire returns from a range of industries. This was used as the basis for the statistical analyses and reworking of the survey tool. One fundamental requirement of this revision is that the statements are meaningful, i.e. there must be a strong conceptual/academic basis for the inclusion of each statement. Furthermore, individual statements are only of use if they have reasonable representation across the distribution of possible responses. Therefore variables with heavily biased response distributions have – by definition – relatively little variance and are likely to be of limited use in terms of distinguishing between strong and weak safety climates. Finally, questions that are unrelated to other questions (i.e. are uncorrelated) will also be of limited use in the final model. The revision of the safety climate tool followed the methodology outlined in Figure 1. There have been three stages in the revision of the original health and safety climate tool (HSE, 1997), namely: removal of statements which fail face validity tests, removal of statements with biased distributions, and removal of statements during exploratory factor analysis. These were then subject to factor analysis to create a comprehensive survey tool.
248
C. Sugden et al. Original Health and Safety Climate Survey Tool 71 Questions, 10 factors
Stage 1 - Face validity removal of 12 questions
Stage 2- Biased distributions Removal of 13 questions
Stage 3 - Exploratory factor analysis Removal of 9 questions Revision of safety climate question set 37 questions remaining
Factor solution for Revised Safety Climate Tool Addition of 3 questions
Piloting of Revised Safety Climate Tool 40 Item Question set Piloting loop 40 question, 8 factor SCT
Revised Safety Climate Survey Tool 40 Questions
Figure 1.
Overview of method.
Stage 1 – Face validity The project team removed statements that were not universally answerable by all respondents, were considered too conceptually similar to others, or were poorly worded. Each of the 71 variables from the original Climate SurveyTool was assessed to determine how well it was able to satisfy these criteria. To achieve this, a statement must be unambiguous in its interpretation, be relevant to the issue of safety climate and be liable to generate meaningful responses. 12 statements were unable to satisfy these criteria. For example: “My job is boring and repetitive” was removed as it was felt that understanding the responses to this question would have nothing meaningful to say about safety climate, and furthermore there are no actions that can be taken to resolve this by an organisation. Whereas, “People here do not remember much of the health and safety training which applies to their job” was felt to be ambiguous. In particular, it was felt that the word “people” could have a number of interpretations
The development of HSL’s safety climate tool
249
from the respondent, their team to other people within the organisation. Furthermore health and safety training could cover either health and safety aspects of their job or specific health and safety training (e.g. risk assessment training), and that there was sufficient uncertainty concerning the question and the meaning of the response. Finally, all question relating to “permits to work”, were removed on the grounds that they were not relevant to the goals of the analysis in that they are not universally answerable.
Stage 2: Removing statements due to biased distributions The second stage of question removal involved looking at the distribution of responses for each of the questions in turn and identifying those with heavily biased distributions. Choosing a critical value for this type of exclusion is not straightforward as there is no objective criterion to measure against. Consequently, a compromise has to be reached between excluding the most heavily skewed variables whilst not affecting the pool of questions too radically by choosing too radical a criterion. In this case, a cut-off point of 70% at either end of the distribution was selected. That is, a variable was excluded if the sum of the “strongly agree” and “agree” categories or the “strongly disagree” and “disagree” categories equaled or exceeded 70% of respondents. In total, 13 statements were removed from the statement pool on this basis. Of these, one had also failed to meet the face validity criteria.
Stage 3: Exploratory factor analysis The previous two stages left 47 statements for inclusion in the analysis. A prerequisite of variables for inclusion in Principal Components Analysis is that they contribute something to that analysis. As the procedure works by pooling shared variance, it requires that each variable has some variance in common with other variables in order that it may be shared. Consequently, any variable that is unable to demonstrate sufficient shared variance should be excluded on the grounds that it adds very little to the analysis. Again, the precise threshold that was chosen was somewhat arbitrary as there is no objective criterion. However, that threshold should not discard too many variables to render the analysis meaningless yet should discard those that contribute the least. A cut-off point of 0.5 was chosen as this represents a point below which less than half of a variable’s variance is included in the analysis. Nine statements displayed communalities less than 0.5 and were removed from the analysis. Interestingly, many of these were statements that had been considered to be marginal on either face validity or distribution stages (but which had made it through that stage). The removal of these 9 statements left a pool of 38 statements for inclusion in the revised safety climate factor structure.
Development of a factor solution for the Revised Safety Climate Tool In order to examine the structure of safety climate with the reduced statement set, the Kaiser-Meyer-Olkin measure of sampling adequacy was used to confirm the
250
C. Sugden et al.
Table 1.
Factor definitions and Cronbach’s alpha.
Factor
Title
Cronbach’s alpha
1 2 3 4 5 6 7 8
Organisational commitment to safety ‘Health and safety’ oriented behaviour Health and safety trust Usability of procedures Engagement in health and safety Peer group attitude to health and safety Resources for health and safety Accident and near miss reporting
0.808 0.815 0.803 0.716 0.715 0.724 0.642 0.599
appropriate approach. A KMO score of 0.962 was achieved, suggesting that the suitability of Principal Components Analysis was satisfied. Given the overall adequacy of the model, the next step was to establish the amount of variance accounted for by that model. Six, seven and eight factor models were explored on the basis of Eigenvalues and the scree plot, as well as the meaningful constructs underlying these factors. In total 6 models were developed using orthogonal and oblique rotation, these were considered by the researchers, and the groupings evaluated to identify which factors best represented the notion of ‘safety climate’. It was felt that the eight factor model was most meaningful and conceptually better reflected current opinions of the key facets of safety culture. The eight factors solution was made up from 37 questions.
Factor reliability and validating the factors The reliability of the factors was assessed to determine their likely future usefulness. This determines the overall “strength” of the scale (i.e. the extent to which the individual questions appear to be measuring the same concept), as well as the contribution of each variable. Table 1 shows the results of the reliability analyses conducted on each of the eight factors. From table 1 it can be seen that the three “best” factors all have high reliability scores and the next three are also acceptable. Factors 1 to 6, therefore, represent reliable scales that should be replicable over time (achieving Cronbach’s alpha exceeding 0.7). Factors 7 and 8 had questionable reliability, however, they have fewer questions contributing to them and the appropriateness of reliability analysis is questionable under these circumstances. A proposed solution was to derive some additional questions that may load onto these factors to “boost” their reliability and pilot them to see how well they perform. Three additional questions were written to boost the reliability of factors 7 and 8. This made a 40 statement question set, comprised of eight coherent factors.
Piloting of revised question set Using this 40 item statement survey tool, a programme of pilot surveys were carried out to ensure statement clarity, highlight ambiguities and raise other issues concerning application of the survey tool.
The development of HSL’s safety climate tool
251
Round 1 – an internal pilot on 20 individuals within HSL was carried out. This highlighted a number of issues, including typos, confusion over use of words (e.g. line manager and supervisor), use of idiom and jargon. Round 2 – the issues raised at Round 1 were explored at a major motor manufacturer when the questionnaire was distributed to around 40 employees. Subsequently semi-structured interviews were held with respondents to gather their perceptions about: • Questionnaire length; • Use of language, including idiom and jargon; • Specific questions identified in the first pilot as ambiguous, confusing, and hard to answer. At this stage a number of modifications were made to the survey tool to clarify some of the areas identified within these 2 pilots. For example a statement was reworded from: My immediate boss is receptive to ideas on how to improve health and safety My supervisor takes on board ideas on how to improve health and safety This change was felt to improve the statement and address the key concept in this instance. Other changes included changing the word ‘seldom’ to ‘rarely’ in several questions, and revising this question: Productivity is usually seen as more important than health and safety Getting the job done is usually seen as more important than health and safety Further pilots were carried out, adopting the same method in the chemicals manufacturing and construction sector. The process again involved semi-structured interviews to confirm some of the modifications and iterations to the question set. Nothing significant emerged from these. Additionally the motor manufacturer involved at the pilot stage rolled out the question set across their large site, and achieved responses from in excess of 600 employees. The piloting process was iterative and confirmatory to ensure that changes did not weaken or significantly change the meanings of statements.
Next steps At the time of writing in the region of 7000 individuals from a range of companies and sectors have completed the revised safety climate tool (SCT). Feedback from these companies has been positive. Companies state that the questionnaire is of the right length (e.g. reduces business burden), written in plain English, provides factors that are meaningful and has prompted organisations to identify solutions to address emerging issues.
Discussion and conclusions From the piloting and use of the revised safety climate tool (SCT) it appears that the technique is valuable to organisations as part of engaging their workforce in the process of safety improvement. Good practice use of the question set, would
252
C. Sugden et al.
involve its application across the organisation, and then carrying out focus groups to identify the reasons underlying respondent’s stated attitudes. This can then be used to develop action plans and obtain staff ownership and engagement. This paper reports on the process of revising a health and safety climate survey tool. Face validity, statistical validity and construct validity were considered in the revision of the safety climate tool. The SCT is comprised of 8 factors and 40 statements, of these 37 statements were retained from the previous model. The goals of the revision were to: • • • •
Reduce duplication and leading questions; Write in plain English; Retain statistical power and feel of the original; Renew the software tool.
To date the project has met the first three aims and work is being carried out on the fourth.
References The B.P. U.S. Refineries Independent Safety Review Panel, (2007), The report of the B.P Refineries Independent Safety Review Panel, http://www.bp.com/liveassets/ bp_internet/globalbp/globalbp_uk_english/SP/STAGING/local_assets/assets/ pdfs/Baker_panel_report.pdf (accessed 19th November 2008). Collins, A and Gadd, S (2002) Safety Culture: A Review of the Literature. Health and Safety Laboratory. Dawson, JF, Gonzalez-Roma, V, Davis, A and West, MA (2008) Organisational climate and climate strength in UK hospitals. European Journal of Work and Organizational Psychology, 17 (1), 89–111. Flin, R., Mearns, K., O’Connor, P. and Bryden, R. (2000). Measuring safety climate: identifying the common features. Safety Science, 34, (1–3), 177–192. Guldenmund, F.W. (2000). The nature of safety culture: a review of theory and research. Safety Science, 34, (1–3), 215–257. HSC (1993) ACSNI study group on human factors. Third report Organising for Safety. HSE Books. HSE (1991) Successful health and safety management. HMSO. HSE (1997) Health and Safety Climate Survey Tool. HSE Books. IAEA (1986) Summary Report on the Post-Accident Review Meeting on the Chernobyl Accident. International Safety Advisory Group, Safety Series 75-INSAG-1 (Vienna: IAEA). Pidgeon, N.F., (1991) Safety Culture and Risk Management in Organisations. Journal of Cross Cultural Psychology, 22, 129–140. Rousseau, D.M. (1988) The construction of climate in organisational research. In International Review of Industrial and Organisational Psychology, Eds. C.L. Cooper and I. Robertson. John Wiley and Sons, Chichester. Sugden, C and Healey, N (2008) Systematic review of key health and safety management literature. Internal report. RSU/08/06.
HUMAN FACTORS INTEGRATION
REPORTING LOCATION AND ENVIRONMENTAL FEATURES WHEN USING METAL DETECTOR OR PROBE IN A SIMPLIFIED MINE DETECTION TASK R.J. Houghton, C. Baber & J.F. Knight EECE, The University of Birmingham, Birmingham, B15 2TT, UK In this paper, the activities involved in searching for buried items with technical aids are explored. The domain of exploration is topical because it can be applied to the search for explosive devices by military and security forces, the ergonomics of which have received relatively little attention. A study is presented which tests the hypothesis that search involves separate cognitive processes for recognising locations and features. It is shown that, while both forms of search tested (i.e., using a trowel or a metal detector) perform similarly when considering location, the recall of features is impaired when using the metal detector. This is interpreted as evidence of different search strategies and information processing in the two conditions.
Introduction Despite its significance to police and security operations, there is relatively little research concerning how people search for buried or hidden objects in the field. In part, this might be because the relevant material could be operationally sensitive and hence, classified. A broader issue is that the psychological literature has located the act of searching largely within the domain of well-defined laboratory paradigms concerning the detection of abstract stimuli amongst distractors or within noise. In the present paper we report an experiment based on memory for buried items that bridges this gap between the laboratory and the field by considering search as an ongoing activity in which participants interact with their environment to discover target objects rather than passively observing. A key research question in this area is how technology might mediate or interact with the tasks of searching and reporting. In particular, we are concerned with the search for buried land-mines (see figure 1). Previous Human Factors research concerning mine detection equipment has suggested that it would be valuable to provide the user with some form ‘map display’ showing the location of objects as they were detected (Herman and Igelias, 1999; Herman et al., 2001). Studies of the use of mine detectors have characterised the task of mine detection as scanning an environment to compose a mental representation of space and the location of mines within it based upon auditory output from the metal detector (Staszewski, 1999; Stazewski & Davison, 2000). This implies that the expert is building a 2D map of the ground, and then interpreting this map against his knowledge of landmine patterns. In an emergency situation in which there is an absence of mine 255
256
R.J. Houghton, C. Baber & J.F. Knight
Figure 1.
Mine Clearance Patrol (image copyright M.O.D).
detection equipment, and/or where it is suspected low-metal mines are present that cannot be detected by metal detectors, an alternative body of technique involves the close physical inspection and probing of the ground. In the present experiment we examine whether these two different techniques might lead to differing forms of mental representation of the searched space, and thus change the quality of a subsequent verbal report. An aspect of mental representation that may be impacted by the degree to which search is mediated by technology and tools is the relative retention of environmental features (that is, the details of the small patch in which a target is located) versus the retention of global target location. Both aspects of a mine or explosive device are necessary parts of a report as they inform how further action can be a taken. Clearly one must know the position of a potential explosive device but knowledge of what is surrounding or covering the object can also have a bearing on how it would be recovered or acted upon. A distinction between spatial and identity representations has been suggested in the memory literature (e.g., Baddeley and Logie, 1999; Smith & Jonides, 1997). From this perspective, it might be argued that some technologies are better able to support one subsystem than the other. In the context of mine-detection, one might argue that a search involving a metal detector might be good for location (because the movement involved in using such a device might involve broad sweeps over local areas), but less good for detecting environmental features (because the sweeping might not support detailed local inspection); conversely, the use of a tool to dig into the ground might be good at supporting detecting environmental features (due to longer time spent at each location), but less good at location (because the inspection of each object might not be remembered as part of a spatial sequence so much as a temporal sequence. In previous studies, Baber and Houghton (in press) found that manual search led to superior recall in object identification over search using a metal detector, but that recall of location was similar for specified targets (although the metal detector condition led to better performance on a task involving the marking of allpossible targets). This implies that the studies which had identified improvements relating
Reporting location and environmental features when using metal detector
Figure 2a.
257
Layout of Markers on the Trays (trays containing metal objects are shaded).
Figure 2b. Trays laid out for experiment (key to letters: B = blue, Y = yellow, R = red, Bk = black). to support for the development of spatial patterns might not be related to the design of the technology per se, but rather to harnessing the ability to process location information.
Method The experiment involved 14 undergraduate students (12 male, 2 female). None had prior experience of the task or the domain. Thus, the experiment is not concerned with the specific abilities involved in mine detection, but addresses the general ability to search with different forms of technology support. The task involved a set of 16 seed trays laid out in a 4 × 4 grid. Each tray was filled with sand in which metal objects were buried in 8 of the trays. Each tray had a marker (3 × 3 cm) with a coloured shape printed on it. The layout of the markers is shown in figure 2. The objective of all participants was to search the trays, as many times as they wished,
258
R.J. Houghton, C. Baber & J.F. Knight
until they were satisfied that they had found all of the metal objects and would be able to report both the location of the objects they had found and the identity of the markers for each object’s location (in terms of colour and shape). Participants were informed that they would be asked to recall one set of this information per trial, i.e., either recall the location of the object or recall the identity of the markers. Participants only completed two searches (one for each form of recall). The reason for this was to minimise the chances that they would simply learn the layout of the trays and rely on recall rather than search. Participants were randomly allocated to one of two conditions: metal detector or trowel. Prior to the trials, participants received an initial explanation on the study. This emphasised that the study was concerned with how well people could remember where they had found buried objects, emphasising that recall could be in terms of location or in terms of some feature in the environment (e.g., grid reference A4 or ‘by the tree’). They were then shown two test trays, one containing only sand and one with a metal object buried in the sand. They were asked to check these test trays in order to get a feel for how the search would be conducted. Participants with the trowel probed and dug in the sand until they were satisfied that an object was, or was not, present. Participants with the metal detector had to position the head of the device on the sand and heard a beep when metal was present. These practise trays provided both groups an opportunity to familiarise themselves with the task prior to the study. When they were satisfied that they understood the aims of the study and the tasks, participants were led to the 4 × 4 grid of trays and asked to search until they would be able to identify where all the metal objects had been hidden. Following this, they were led away from the grid and asked to recall either location of objects or identify of markers. Recall tasks were counterbalanced across participants. Next the search task was repeated and the other recall task performed. The dependent variables for this study were: time to complete the search task; number of checks made; recall of location; recall of marker identity; subjective rating of workload (using NASA-TLX). Pair-wise comparisons were made between conditions using t-tests.
Figure 3.
Checking the trays using trowel (on left) or metal detector (on right).
Reporting location and environmental features when using metal detector
259
Results The results can be divided into three sections: search activity, reporting performance and workload.
Search activity In terms of time spent conducting the search, there is a small (but non-significant) difference between the conditions. On average, participants in the Trowel condition spent 108 (±35)s on the task, and participants in the Metal Detector condition spent 84 (±29)s. However, there was a difference in the number of checks made. On average, participants in the Trowel condition made 16 (±0.5) checks, whereas participants in the Metal Detector condition made 28 (±6) checks. This difference is statistically significant: t (12) = 5.2, p = 0.0002. If we crudely approximate a search per location as Mean time/Checks made, then we can see that the Trowel condition results in around twice as time spent per check, i.e., 6.7 (±2.2)s per check versus a time of 3.0 (±0.7)s per check for the Metal Detector. This difference is statistically significant: t(12) = 3.729, p = 0.0029. A comparison of the average number of checks for each tray further supports the claim that the metal detector condition led to a higher number of checks than the trowel. However, as indicated by the figures in bold in figures 4a and 4b, there does not seem to be any relation between number of checks and likelihood of finding metal in a tray.
Reporting performance Recall performance was measured on two dimensions: location and shape. In terms of location, there was no difference between conditions, i.e., recall of location for the Trowel condition had a mean of 4.9 (±2.1) correct items per trial and the Metal Detector condition had a mean recall of 5 (±2.8) items per trial. The location data were also analysed in terms of the participants’ ability to correctly trade-off recall of the location against erroneous recall. This was analysed as a form of signal detection (although, of course, we are not analysing the detection of the targets but the ability to recall the found targets which is quite a different question). Calculating d as z(FA) − z(H), we get a d for the Trowel condition of 1.004, and for the Metal Detector condition of 0.912. This demonstrates that the performance in the two conditions was very similar in terms of recalling the location of the targets. There was a noticeable difference between conditions in terms of the recall 1.4 1.4 1.4 1.5
1.4 1.5 1.5 1.6
1.4 1.6 1.5 1.6
1.5 1.5 1.6 1.5
Figure 4a. Mean checks per tray (Metal Detector).
1 1 1.1 1.1
Figure 4b.
0.9 1 1 1.1 1.1 0.9 1.1 0.9 1 1.1 1 0.9
Mean checks per tray (Trowel).
260
R.J. Houghton, C. Baber & J.F. Knight
Table 1.
Comparison of Recall Performance for the two Conditions.
TROWEL
Target
No Target
METAL
Target
No Target
Recall No Recall
0.64 0.36
0.29 0.71
Recall No Recall
0.68 0.32
0.29 0.7
20 18 Metal Detector 16
Trowel
14 12 10 8 6 4 2 0 Mental
Figure 5.
Physical
Temporal Performance
Effort
Frustration
Mean NASA-TLX ratings across the two conditions.
of the shape. The Trowel condition had a mean recall of 5.1(±1) items per trial, while the Metal Detector condition had a mean recall of 3.0 (±1.3) items per trial. This difference was statistically significant – t(12) = 3.5324, p = 0.0041.
Subjective rating of workload A mixed Analysis of Variance (workload dimension × condition) was performed and revealed a signficiant main effect of workload dimension [F (5,60) = 6.221, p < 0.001] but not of condition. Inspection of figure 5 shows that the individual differences between participants (as shown by the standard deviation) was sufficient to obscure any differences between condition. However, we note small differences on the Physcial and Effort scales (which could be attributed to the need to bend and crouch when using the trowel, rather than simply walk around as with the metal detector), and also on the Mental and Frustration scales (which could imply that the search task was perceived as a little easier with the metal detector).
Conclusions Previous work into the ergonomics of mine detection had tended to focus on supporting spatial ability. The relative over-sampling of locations by the metal detector users in the experiment supports Stazewski’s account that some form of sequence
Reporting location and environmental features when using metal detector
261
construction (possibly physically rehearsed using the metal detector itself as much as held in memory) is occurring: “. . . he located mines by sequentially finding edge points defined by the detector’s auditory outputs, holding them in memory and creating spatial patterns in his mind’s eye by linking contiguous edge marker locations” (Stazewski, 2007). However, this did not eventually lead to better spatial report of target locations than in the trowel condition (a caveat to this being that the spatial array in the present experiment only consisted of 16 cells). In terms of report of environmental features, the trowel users whose search was characterised by a slower, more effortful progress through the search array, showed superior performance compared to metal detector users. One interpretation of this is that feature identification is not well supported by the metal detector precisely because it engenders ‘scanning’ behaviour, which might suggest a possible problem in terms of conducting search of the environment and the location at the same time. One could suggest that, to some extent, military practice recognises this potential difficulty because in ideal circumstances it tends to divide search for mines into separate phases of detector-based search, marking and finally inspection. The present results suggest that in the development of future technologies to support search, consideration should be given to how the output of various devices can be combined in such a way as to streamline this process by either supporting both location and featural representation (which may not be possible if trade-offs between the two can not be reconciled) or else to automate either the representation of space (through positioning systems) or the representation of environmental features (through digital imaging) to allow the operator to allocate their attention to a specific dimension of an ongoing search task.
References Baber, C. and Hougthon, R.J., in press, Transitional information in a simplified mine detection tasks, The European Chapter of the Human Factors and Ergonomics Society Annual Meeting, TNO Soesterberg, October 2008 Baddeley, A. and Logie, R.H., 1999, Working memory: the multiple component model, In P. Shah and A. Miyake (eds) Models of Working Memory, Cambridge: Cambridge University Press, 28–61 Herman, H. and Iglesias, D., 1999, Human-in-the-loop issues for demining, In A.C. Dubey, J.F. Harvey, J.T. Broach and R.E. Dugan (eds.) Detection and Remediation Technologies for Mines and Mine-like Targets IV, Proceedings of SPIE, 3710, Bellingham, WA: SPIE, 797–805 Herman, H., McMahill, J.D. and Kantor, G., 2001, Enhanced operator interface for hand-held landmine detector, In A.C. Dubey, J.F. Harvey, J.T. Broach and R.E. Dugan (eds.) Detection and Remediation Technologies for Mines and Minelike Targets VI, Proceedings of SPIE, 4394, Bellingham, WA: SPIE, 844–851 Smith, E. E. and Jonides, J., 1997, Working memory: A view from neuroimaging. Cognitive Psychology, 33(1), 5–42 Staszewski, J.J., 1999, Information processing analysis of human land mine detection skill, InA.C. Dubey, J.F. Harvey, J.T. Broach and R.E. Dugan (eds.) Detection
262
R.J. Houghton, C. Baber & J.F. Knight
and Remediation Technologies for Mines and Mine-like Targets IV, Proceedings SPIE 3710, Bellingham, WA: SPIE, 766–777 Staszewski, J. J., 2007, Spatial thinking in the design of landmine detection training. In G.L. Allen (Ed.) Applied Spatial Cognition, 231–267. New Jersey: Lawrence Erlbaum Associates Staszewski, J.J. and Davison, A., 2000, Mine detection training based on expert skill, In A.C. Dubey, J.F. Harvey, J.T. Broach and R.E. Dugan (eds.) Detection and Remediation Technologies for Mines and Mine-like Targets V, Proceedings of SPIE, 4038, Bellingham, WA: SPIE, 90–101
PREDICTIVE OPERATIONAL PERFORMANCE (PrOPer) MODEL K. Tara Smith HFE Solutions, Dunfermline Fife, Scotland Operational Performance Assessment or Operational Analysis is a key tool that can provide vital insight to force composition, interaction of new equipments, strategy, etc. However to date the human component has to the most part been filled by assumptions and rough measurements from similar tasks. This in turn means that issues like training, skill level, human workload and human error are not accounted for in the results. In recent years there have been a number of models developed that start to address this gap between Human Factors and the Operational Analysis fields. Two key developments are: • the STORM (Socio-cultural Teamworking for Operational Research Models) Model developed by DSTL and • the 4CCWM (For Close Combat Workload Model) developed by HFE Solutions This paper presents a conceptual framework that integrates these two models and discusses the integrated model which provides an initial pathway to predict operational performance from task and team composition. This new PrOPer model provides a coherent framework that allows an analyst to consider the consequence of workload in teams and scenarios. It addresses the major components of the human/task attributes and predicts the human performance impactors, thereby deriving a resultant performance and accuracy prediction. It is hoped that this approach will eventually remove the need for the OA models to make assumptions regarding human elements.
Introduction In the past, there has been much dissatisfaction at the lack of Human Factors models to feed into Operational Analysis (OA) studies. There is a recognised lack of human performance models that can be directly applied to OA due to the level of fidelity of the models. This paper describes one approach to address this longstanding issue. The development of this Predictive Operational Performance (PrOPer) model was inspired by the realisation that the STORM model and the 4CC Workload Model, although they address different problem areas, take a fundamentally similar approach to describing the task drivers and personnel profiles, utilising similar data and techniques. 263
264
K. Tara Smith
This paper presents an overview of the resultant model that combines elements of these two fundamental pieces of work and proposes that this approach can be used to supplement an OA assessment. This project resulted in a demonstrator of the model: this demonstrator did not provide the full functionality but was constructed in a manner that would facilitate the easy implementation of additional elements and features. The combined approach embodied elements of a number of other models and approaches. These include; • Wickens’ multiple resource theory • McCracken and Aldrich’s visual, cognitive, auditory, and psychomotor (VCAP) theory • Hopkins’ training methodology • Smith’s finite time element approach • NASA TLX assessment approach • Tuckman’s team maturity process • Noble’s knowledge enablers • Mathieson and Dodd’s model of organisational and social factors. Put simply, the objective was to be able to describe the social and cultural impactors, training levels, team structure, tasks and task demands to produce a model which can then predict; • • • •
Likely delays due to high workload Risk of human error A profile of high workload “hotspots” Task sensitivity to workload.
This prediction allows the examination of variations in team make-up, training levels and task allocation.
Context In any safety-critical industry (which includes the military environment), tasks often have to be performed within strict timescales. This means that the workload on the individual can be critical. The nature of human factors is not just looking at equipments and environments; it is understanding the fundamental capabilities of the human in the system and relating those to the demands of their tasks in the context of the environment in which they have to work. The PrOPer model allows the analyst to represent the task demands caused by the equipment and the environment and their dynamic nature, which can be driven by external events and relates it directly to the capabilities of the user. Other workload approaches have tackled this area by modelling the workload associated with the equipment: the approach taken in this model follows Hilary Putnam’s paradigm which is that weather systems are mainly driven by the inherent properties of water molecules, however studying water molecules does not allow
Predictive operational performance (PrOPer) model
265
you to predict the weather. The model allows the representation of an abstract level of workload for a whole team as well as for an individual person. Wickens’ multiple resource theory suggests that if this workload can be shared between the different cognitive resources then the accumulative workload is less likely to reach a critical threshold: the development of this model is intended to predict likely workload “hotspots” for the different cognitive resources thus allowing the possibility of mitigation strategies such as moving a task from visual to auditory or adding additional personnel. The model embodies Wickens’ multiple resource model to allow these workload aspects to be investigated. Tuckman’s work regarding the dynamics of teams and the subsequent work carried out by Mistry and Waters have created predictive models that allow the modelling of different scenarios and the consequences of changes to the team make-up. PrOPer embodies the STORM model, which was created to model the consequences of the make-up of the team. One current theme within military research is to do “better things” as well as “doing things better”: i.e. as well as improving current practices and equipments (doing things better) to introduce new and innovative approaches to solving the same problems (doing better things). This inevitably leads to requiring a better understanding of how things function and the underlying relationships between individual tasks and the overall performance of the manned system(s). Additionally, with the increasing costs of development, it is beneficial to be able to predict the consequences of decisions regarding equipment, manpower, etc., without committing to expensive development programmes. Therefore the model needs to represent the full operational context: i.e. hours or days of activity and 10s or 100s of individuals. The creation of predictive OA and HF models, if utilised as part of the procurement and investigation strategies, can have a major effect on identifying and fine-tuning strategies that improve manpower profiles, training, equipment and other cost drivers. As the model is data-driven it allows the same assessment to be applied to different solutions, providing better confidence in the comparative results. Currently the MoD requires a programme to be assessed through a series of financial gateways: in effect a “through-life cost versus operational effectiveness” trade-off. Although the human component of that trade-off is significant to determining the success of the programme, currently little data is transferred up from the human factors work to that level of decision-making. This model provides easily understood comparative data to be fed into the Government assessment/trade-off process, whilst still providing a view of the detailed human-related causal factors. Initial validation of the results of the demonstrator was conducted as part of the NEC4CC programme: the user trial was conducted over five days, however only involved three subjects. The trial analysed a 12-hour scenario representing a full Company (60–200 soldiers) and focussing on the three Company Commanders. Data collected during the trial was analysed against the model of the same scenario and concluded that the model represents over a 90% approximation to the study data. Further validation trials are planned both within NEC4CC and on other MoD research projects. This paper does not cover the validation activities or the additional development plans.
266
K. Tara Smith
The PrOPer model As there are still gaps in human factors knowledge, this model was constructed in such a way as to allow extensions to include new research without compromising the underlying data structures. Wherever practicable in the model described below, where research was not sufficiently mature to provide a capability of modelling that aspect, the aspect was represented in the model by a data structure that could in the future be acted upon by a model. An example of this is how fatigue affects the workload threshold.
Background The two basic pieces of work that fed into PrOPer were: • STORM: STORM models the impact of social and cultural factors upon teamwork. It is designed to improve the state of the art of modelling military command and control, in particular Headquarters (HQ) modelling, in support of Level 2 Operational Analysis (OA). Level 2 OA provides support to Capability Management and must operate at the system level of abstraction, working with models which explain the effects of interventions in equipment and other Lines of Development at this level. • 4CCWM: The Information Overload Study was part of the Network Enabled Capability for Close Combat (NEC4CC) research programme. Its objective was to create a individual model of workload combined with a team-level model which identifies the information needed to be passed around the team, as currently there are no formal models of the cognitive and physical workload demands on people within a close combat operational situation.
Conceptual framework The first step in creating the model was to create a top level conceptual framework to allow the description of the components of both STORM and 4CCWM within the context of operational performance. Conceptually, the framework is divided into four blocks: • Elements of Change: changes to equipment, training, etc will impact on the human/task attributes. • Human/Task Attributes: modifications to the individual aspects of the experience and capability of the humans will result in changes to the human performance impactors. • Human Performance Impactors: the performance of the humans who make up the teams and fulfil the tasks needs to be understood as their variation impacts directly upon the overall performance of the manned system. • Operational Analysis: focuses on overall performance, based on time to complete and/or accuracy of the predicted results. It needs to know about the people/teams and their logical deployment (OrBAT) and the external events that affect the operation.
Predictive operational performance (PrOPer) model Human / Task Attributes
Human Performance
267
Operational Analysis
Impactors
Equipment Design
Knowledge
Group Behaviour
Doctrine
Target Audience
Individual Behaviour
Training Design
Teamwork Performance
Attitudes Task Performance
Event Time Sequence
Manpower Tasks
Personnel Selection
Scenario Enviroment
Environmental Impactors on Performance
OrBat
Workload
Human Error Impactors on Performance
Success Prediction
Task Performance
Equipment Protective Equipment
Figure 1. Top level conceptual framework. By modifying the elements of change, the human/task attributes are altered, which affects the human performance impactors and thus impacts upon elements of the operational analysis. For example, will giving soldiers a better radio (changing the equipment design) improve the overall performance of the regiment? The conceptual framework provides an overview of the human factors domains that are utilised to generate the base-level data, how these impact on a programme and how they are represented in the operational analysis trade-off studies. The conceptual model includes a representation of the following elements to be able to answer and correspond to OA-related models in estimation of human workload and team capability; • Scenarios, goals and actions: to allow representation of different military vignettes • The team workload: to allow prediction of team performance • The individual’s workload: to allow prediction of individual’s performance • The components of the actions: to provide a generic set of components that can be mapped to any task • The demands of those action components: to be able to calculate the workload • The teams that are going to undertake the operation: to be able to represent different teams with different experience levels running through the same scenario • The characteristics and capabilities of the individuals in that team: to model their ability to undertake the activities • The overall team performance: to provide a standard metric of time and number of delays. These aspects are captured within the three basic components of the overall system model.
Top level architecture The following diagram shows the top level architecture of the model: i.e. the three basic components.
268
K. Tara Smith Individual Task List
Perceived Risk
Scenario, Goal & Task Level
Performance & Workload Level
Team Level Training Level of Individual
Resultant Task Workload Team
Team Maturity Level
Figure 2.
Resultant Task Delay Team
Components of the system model. S c e n a rio,Go a l & Ac tio n s Activity
Scenario Components Goal Triggering Events
Individual Task List
ORBAT Mission Command Mission–– Command Intent
Team
External Events
Individual
Operational Context Perceived Risk
Figure 3.
Scenarios, goals and actions.
The following sections describe each of these top-level components in turn.
Scenario, Goal and Task Level For modelling purposes, an assumption was made that a scenario can be fully described by a set of triggering events that happen at timed intervals throughout the scenario. The triggering events initiate goals which in turn are represented by a set of activities that have to be completed to satisfy the overall goal. Additionally, conditions can be set to ensure that an activity cannot be started prior to another activity being completed. As activities are assigned to individuals, this structure allows for a triggering event to initiate team-wide goals. As the model runs, it dynamically creates a task list for each individual. In the demonstrator, the perceived risk of the operation is being entered as a settable value and is not being derived from any aspects of the overall operational scenario.
Team Level The STORM model takes parameters that represent aspects of team dynamics such as skill levels, training levels, cultural coherence, etc., amalgamates these into intermediate values such as willingness to co-operate and from there derives what overall state the team is operating in, in terms of forming, storming, etc. This study
Predictive operational performance (PrOPer) model Forming
Storming
Norming
269
Performing
Activity Awareness
Goal
External Situation
Role, Task
Current Task
Others
Plan Assessment
Team Business Trust
Relationships
Decision Drivers Mutual Understanding
Task Skills Current Task Work
Team Initiation Training
Experience
Socialisation
Figure 4.
External Info Feed
Co-location
Underlying structure of STORM.
has taken the position that the overall team performance (team maturity) is provided by an instantiation of the STORM model within this overall conceptual model. The current instantiation of the STORM model only captures and models the characteristics of an entire team and does not consider the impact of the training level of an individual within the team. Thus there is an unmodelled relationship between the sum of the individuals’ training levels and the maturity of the team. This aspect is not modelled in the demonstrator or the conceptual model as no previous research work has been identified that relates these two elements, but is represented within the model by an additional criterion that indicates the training level of an individual. The conceptual model recognises that some activities can be dynamically allocated to different individuals within the team. It implements this by providing a simple look-up table that links an activity to “can be undertaken by”. This approach allows for the simple cases where an activity can be allocated to one or more individuals, however there are other factors that affect selection, for example, trust by the Commander, fatigue, etc. The demonstrator does not implement this aspect, however, the structure and data to allow it to be implemented in future versions is provided.
Performance and Workload Level The conceptual model adopted a slightly modified version of the structure utilised in the 4CCWM. The generic activity components are structured into four “banks” of activities that can be mapped to any individual task. These banks represent different aspects of the overall job, such as team-working and risk-monitoring. The conceptual model starts by representing the activity demands as a matrix of activity demands versus generic activity components, based on an expanded NASA TLX assessment method. Each bank of the generic activity components are then operated on to provide a resultant workload for each activity bank.
270
K. Tara Smith
Following the above integration, the model filters and scales the workload by what aspects of that activity are currently active within a specific time slot, then sums the workload from each activity assigned to that individual within that time slot. At this point, an assessment is carried out as to whether that individual has exceeded their workload threshold. The workload threshold is where the combination of task demands that are allocated to an individual are greater than the capability of that individual to perform the tasks in the time available. If the threshold is exceeded, then the lowest priority activity is pushed on to the next time slot. This approach is known as a finite time element approach. This mechanism only represents a simple “delay a task” coping strategy and does not allow for other coping strategies that an individual might adopt, such as passing off a task to a colleague, limiting the scope of a task in order to complete it quickly, etc. Additionally, it must be recognised that the workload threshold is dynamic and variable and that this should be taken into account when reviewing the final results of the model. The team workload is a composite of the workload of the individuals within the team and any composite delays to the completion of activities caused by an individual’s workload threshold. Note: as the workload threshold is decreased, the amount of time that the team is operating in and around their threshold will increase and the number of delays will also increase. It is therefore suggested that the model may be run at multiple thresholds to determine the sensitivity of the threshold to the resultant delays.
Results of the model (Outputs) The outputs of the modelling process can be examined at two levels: • The macro (operational analysis) level where the model can provide an overall assessment of how long an operation may take (including delays caused by high workload) and its subsequent measures of effectiveness. • The micro (human factors) level where the model can be used to examine the causes and sensitivity of workload levels. It is important to remember that the results from any modelling activity such as this must be interpreted and not just taken at face value. The following list provides examples of the interpretation that must be applied to the results for each of the desired outputs: • Likely delays due to high workload: the model reports tasks that have been delayed due to high workload for each timeslot calculated. The analyst needs to question whether the priorities at that point in time were appropriately modelled and whether there would be any other coping strategies that could be adopted by the individual, other than just delaying the task. • Risk of human error: although very high or very low workload are often considered the main drivers of human error, it is well established that when an individual considers the actual risk (safety or risk to operation) to be high, they take more care and adopt robust procedures to reduce the instances of uncorrected human error.
Predictive operational performance (PrOPer) model
271
• Task sensitivity to workload: this is in itself an interpretation of the two elements above, considered in the light of variations in workload threshold values. • A profile of high workload “hotspots”: collapsing workload into a single figure can lose critical information. For example, during a trial of the workload model it was found that although visual workload was often very high, it was always additional demands in linguistic/cognitive reasoning that pushed an individual over the workload threshold. For this reason, the model reports on five workload indices, which are tabulated against each other to establish the relationships between them for each individual and each timeslot. These indices are; – Perceptive Index – indication of spatial reasoning – Physical Index – indication of psychomotive processing – Cognitive Index – indication of linguistic reasoning – Demand Complexity Index – compilation of all task demand scores – Distractive Index – all task sensory demands.
Conclusion Current OA models tend to represent the human element in a simplified smanner: the PrOPer model, if used alongside an OA model, would allow a more realistic representation of that human component. This operational performance prediction model could potentially form one component of an overall modelling strategy. It forms the first practical integration of human factors data into an OA-style environment. This work could not have been carried out without the support of Dr Beejal Mistry of DSTL.
References Dr Beejal Mistry and Dr Mike Waters, March 2007, Social Networks & Culture in HQ Modelling. Calibration and Integration of STORM with extant C2 model DARNOS. Graham Mathieson, John Holt (HVR), Beejal Mistry, Neil Verrall, June 2004, Building Effective Representations of Social Networks and Culture in Models of HQ. Graham Mathieson, Beejal Mistry, Mike Waters, 2005, Coping with Social and Cultural Variables in C2 Modelling for Networked Enabled Forces. Smith, K Tara, November 2006, NEC for Close Combat WP 1.6.3 Information Overload Interim Report 72/06/R/300/I Issue 2. Wickens, C.D., 1984 Engineering Psychology and Human Performance, Harper Collins Pubs. McCracken, J.H. and T.B. Aldrich, 1984, Analysis of selected LHX mission functions: Implications for operator workload and system automation goals, Army Research Institute Aviation Research and Development Activity.
DISTRIBUTED SITUATION AWARENESS: APPLICATIONS AND IMPLICATIONS IN DEFENCE P.M. Salmon1 , N.A. Stanton2 , G.H. Walker2 & D.P. Jenkins2 1
Human Factors Integration Defence Technology Centre (HFI-DTC) Human Factors Group, Monash University Accident Research Centre, Monash University, Victoria, Australia 2 Transportation Research Group, University of Southampton, School of Civil Engineering and the Environment, Highfield, Southampton, SO17 1BJ, UK Situation awareness is a critical consideration in defence system design and evaluation. Despite almost two decades of research in the area, however, attempts to describe team situation awareness remain ambiguous. This article presents a model of distributed situation awareness, which uses the sub-concepts of compatible situation awareness and situation awareness transactions, to explain how teams develop and maintain situation awareness during collaborative activities. In addition, an accompanying modelling approach, the propositional network methodology, is outlined and demonstrated. Following this, a case study undertaken in the land warfare domain, involving a situation awareness-based analysis of a newly developed digital mission support system is summarised. In closing, the implications of both the model and case study findings are discussed with regard to the design of future defence systems.
Introduction The popularity of Situation Awareness (SA), the concept that focuses on how operators in complex systems acquire and maintain awareness of ‘what is going on’ (Endsley, 1995a), continues to rise. Notwithstanding this, and over two decades of scientific investigation, SA remains a contentious topic; the inescapable conclusion arrived upon by the authors following reviews of SA theory (Salmon et al., 2008) and measures (Salmon et al., 2006) is that there are still significant issues that need to be confronted. Of most relevance given the increasing presence of teams within complex sociotechnical systems (Fiore et al, 2003) are the limitations of existing models (e.g. Endsley, 1995a; Smith & Hancock, 1995 etc) and measures (e.g. Endsley, 1995b) when applied to collaborative systems. The aim of this article is to describe new approaches to both problems, including a recently developed distributed cognition based model of SA in collaborative environments and an accompanying modelling approach. Both are subsequently demonstrated using a case study example from the defence domain and, in closing, the implications of the model and case study findings for the design of defence systems are discussed. 272
Distributed situation awareness
273
Distributed situation awareness Concepts such as shared SA (e.g. Endsley & Robertson, 2000) and mutual awareness (Shu & Furuta, 2005) remain ambiguous. Certainly, shared SA seems to be a blunt characterisation that leaves key questions unanswered. For example, does shared SA mean that team members understand elements of a situation in exactly the same manner? Are team members with access to the same information really arriving at the same situational picture? How do different team member roles, tasks, experience and schema map onto the idea of shared SA? Can team SA be satisfactorily explained by looking at SA in the heads of individual operators? All are difficult, but pertinent questions and existing models do not seem to contain sufficient answers. As a corollary, many in the field have articulated the need for further explanation of team SA (e.g. Artman and Garbis, 1998; Gorman, Cooke & Winner, 2006; Salmon et al, 2008 etc). Following Artman & Garbis (1998) call for distributed cognition (e.g. Hutchins, 1995)-based models of SA for collaborative systems, Stanton et al. (2006) proposed the foundations for a distributed cognition-based theory of SA. These foundations were built upon by Stanton et al. (2009) who outlined a model of DSA, developed as a result of applied research in a range of military and civilian command and control environments. Briefly, Stanton et al’s model is underpinned by four theoretical concepts: schema theory, genotype and phenotype schema, Neisser’s (1976) perceptual cycle model of cognition and, of course, Hutchin’s (1995) distributed cognition approach. The model takes a systems perspective approach to SA and views SA as an emergent property of collaborative systems; SA therefore arises from the interactions between agents; it is associated with individual agents but it may not reside within them as it is borne out of the interactions between them. At a systemic level, awareness is distributed across the different human and technological agents involved in collaborative endeavour. Stanton et al. (2006; 2009) suggest that a system’s awareness comprises a network of information on which different components of the system have distinct views. Scaling the model down to individual team members, it is suggested that team member SA represents the state of their perceptual cycle (Neisser, 1976); individuals possess genotype schema that are triggered by the task relevant nature of task performance, and during task performance, the phenotype schema comes to the fore. It is this task and schema driven content of team member SA that brings the shared SA (e.g. Endsley & Robertson, 2000) notion into question. Rather than possess shared SA (which suggests that team members understand a situation or elements of a situation in the same manner), the model instead suggests that team members possess unique, but compatible, portions of awareness. Team members experience a situation in different ways as defined by their own personal experience, goals, roles, tasks, training, skills, schema and so on. Compatible awareness is therefore the phenomenon that holds distributed systems together (Stanton et al., 2006; 2009). Each team member has their own awareness, related to the goals that they are working towards. This is not the same as other team members, but is such that it enables them to work with adjacent team members. Awareness is not shared; instead the situation is viewed differently based on these personal and task-related factors. Each team members’
274
P.M. Salmon et al.
SA is however compatible since it is different in content but is compatible in that it is all collectively required for the system to perform collaborative tasks successfully. The question remains as to how DSA is built between team members? Of course, the compatible SA view does not discount the sharing of information, nor does it discount the notion that different team members have access to the same information; this is where the concept of SA ‘transactions’applies. Transactive SA describes the notion that agents within collaborative systems enhance the awareness of each other through SA ‘transactions’. A transaction in this case represents an exchange of SA information from one agent to another (where agent refers to humans and artefacts). Team members may exchange information with one another (though requests, orders and situation reports); the exchange of information between team members leads to transactions in the SA being passed around; for example, the request for information gives clues to what the other agent is working on. The act of reporting on the status of various elements tells the recipient what the sender is aware of. Both parties are using the information for their own ends, integrated into their own schemata, and reaching an individual interpretation. Thus the transaction is an exchange rather than a sharing of awareness. Each agents SA (and so the overall DSA) is therefore updated via SA transactions. Transactive SA elements from one model of a situation can form an interacting part of another without any necessary requirement for parity of meaning or purpose; it is the systemic transformation of situational elements as they cross the system boundary from one team member to another that bestows upon team SA an emergent behaviour.
Modelling distributed situation awareness: the propositional network approach Viewing SA as a systems level phenomenon has interesting connotations for its assessment. Individual-based SA measures such as SAGAT (Endsley, 1995b) are not applicable since they focus exclusively on the awareness ‘in-the-head’ of individual agents and overlook the interactions between them. Instead what is required is an approach that is able to model SA from a systems perspective, including the information that is distributed around the system, the usage of this information by different agents, how the information is combined together to form ‘awareness’ and the SA-related interactions between system elements. Accordingly, the propositional network methodology (Anderson, 1983) is proposed as a way of modelling a systems SA (networks have been used by cognitive psychologists to represent knowledge since the 1970s e.g. semantic networks). A propositional network is essentially a network depicting the information underlying a system’s awareness and the relationships between the different pieces of information. DSA is represented as information elements (or concepts) and the relationships between them, which relates to the assumption that knowledge comprises concepts and the relationships between them (Shadbolt & Burton, 1995). Propositional networks can be constructed from a variety of data sources, including observational and verbal transcript data, cognitive task analysis data, task analysis data or data derived from work-related artefacts such as Standard Operating Instructions (SOIs). Initially, concepts, or information elements, followed by
Distributed situation awareness
275
the relationships between them are defined. This involves undertaking a simple content analysis on the data and extracting keywords. These keywords represent the information elements, which are then linked based on their causal links during the activities in question (e.g. contact ‘has’ heading, enemy ‘knows’ plan etc). The output is a network of linked information elements; the network contains all of the information (and the relationships between the different pieces of information) that is used by the different agents and artefacts during task performance and thus represents the system’s awareness. Thus, the information elements related to DSA, the relationships between them and also the usage, ownership and sharing of these information elements as the scenario unfolds over time can be defined. To demonstrate, Figure 1 presents an extract of a verbal transcript collected during a study of DSA during land warfare activities; the keywords extracted from this transcript via content analysis are highlighted in bold and the ensuing propositional network is presented on the right hand side of the Figure.
Defence domain case study We have recently been involved in various naturalistic studies within the defence domain, including the land warfare domain (see below), the military aviation domain, the naval warfare domain (Stanton et al., 2006) and the multinational warfare domain. For the purposes of this article we focus on one such study, which involved the analysis of land warfare missions using a recently developed digital mission support system (See Stanton et al., 2008). The land warfare study focused on a new digitised battle management system that provides support for battlefield planning and execution tasks. The system provides users with various tools to support the land warfare mission planning and execution process, such as mission analysis, decision support overlay matrix, and synchronisation matrix construction tools and a real-time Local Operational Picture (LOP) display presenting friendly and enemy location and movement information and also various collaborative tools such as instant messaging and email facilities. The aim of our analysis was to assess the impact on DSA that the new mission support system had during mission planning and execution activities. The study was undertaken during a recent operational field trial of the digitised system involving a fully functional Division, Brigade and BattleGroup (BG) undertaking missions using the new system. Analysts located at the Brigade and BG HQ observed mission planning and battle execution activities over the course of the three-week trial. Initially, an SA requirements analysis was undertaken using HTA and interviews with subject matter experts. The findings indicated that the SA requirements of the different Brigade and BG staff were distinct and compatible, rather than shared. The nature of the land warfare mission planning and execution ‘system’ is such that each ‘cell’ within the Brigade and BG has very different roles and thus different, but compatible, SA requirements. As a result, even when team members have access to the same information, they are typically using it very differently and in
276
P.M. Salmon et al.
Figure 1. Propositional network example; figure shows verbal transcript and the resultant propositional network that is constructed based on the identification (via content analysis) of keywords from the verbal transcript data. conjunction with different pieces of information. As a corollary of this, SA across team members is very different in nature but is compatible in that it fits together to enable planning and mission execution activities to be undertaken successfully. To demonstrate, we can look more closely at the mission-planning phase, which involves the Brigade and BG working through seven planning questions in order to develop, select and refine appropriate courses of action. During question 1 (the battlefield area evaluation phase, i.e. what is the enemy doing and why?), the engineer cell is primarily concerned with developing SA the ground, friendly and enemy forces use of the ground and the impact that the ground is likely to have on friendly and enemy operations. The intelligence cell, on the other hand, is primarily concerned with understanding the threat posed by the enemy, including the enemy’s capability, strengths and weaknesses, doctrine and subsequent modus operandi. Thus even when both team members have access to the same information regarding the battlefield area and the enemy (e.g. terrain, enemy location, movements and formation), they use and view the information in a very different manner; it is the relationship between concepts that makes up their distinct SA. There awareness is however compatible since it connects together in order to allow production of the question 1 products; each cell’s awareness of the battlefield, the enemy and the level of threat conjugates together to form the question 1 output. It is this unique combination of information elements and the relationships between them that makes
Distributed situation awareness
Engineer view
Has
Intelligence view
ts
effec
Impact
Ground
ws
Has
Kn o
ws Kn o
Friendly operations
Has
Doctrine
Ground
on
Has
277
affec
ts
Capability
ef fe c
Knows
Knows
Has
Enemy operations
Enemy Has
Use of ground
Has
Is
s Ha
Enemy
ts
Has
Threat level
Has Is Strengths
Is Weaknesses
Figure 2. Extract of propositional networks showing Engineer and Intelligence cells differing views on enemy and ground information elements; figure demonstrates how each component is using the information for their own ends and how its subsequent combination with other information makes each view unique. each team member’s SA compatible and not shared; the very fact that an actor has received information, acted on it, combined it with other information and then passed it onto other actors means that its interpretation changes per team member; this represents the transaction in awareness referred to earlier. The engineer and intelligence components views on the enemy and ground during question 1 are represented in Figure 2. For the DSA analysis analysts directly observed the planning and battle execution activities within the Brigade and BG HQ. Propositional networks were developed based on verbal transcripts, HTA and live observation of the activities undertaken. The DSA analysis of the planning and battle execution activities observed provided compelling evidence of the impact on DSA that the new digitised mission support system. Firstly, compatible SA requirements were not supported in any way by the digital system; the system provided the same tools, functionality, displays and information regardless of who was using it. Secondly, due to problems with the digitised system (poor usability, convoluted processes etc), the teams involved continued to use the traditional paper map processes in order to support and supplement the new digitally supported process. From this it was concluded that the mission support system did not adequately support the acquisition and maintenance of DSA during the activities observed; rather, a combination of the paper map process and the new electronic systems was used. Thirdly, there were many instances in which the SA-related information presented by the digital mission support system was in fact inaccurate and was not compatible with the real state of the world. This was particularly problematic during battle execution, where the information presented
278
P.M. Salmon et al.
on the Local Operational Picture was often out of date or spurious. For example, due to data bandwidth limitations voice transmission was given precedence over global positioning data regarding the locations and movements of entities on the battlefield and as a result of, contact reports and positional information was presented on the LOP often up to twenty minutes late. Fourthly, as a consequence of the problems discussed above, users reported low levels of trust in the SA-related information presented to them by the electronic system; as a result users often took measures to clarify the accuracy of the information (e.g. requests for clarification of location and status reports). This served to add to the planning and execution process and also adversely impacted the tempo of operations.
Discussion The concept of DSA has significant implications for the design of complex collaborative systems, such as digital mission support tools. For example, rather than encourage shared perspectives on a situation (which assumes that team members require identical awareness of shared situational elements), DSA suggests instead that the primary goals of digitised mission support systems should be to support the distinct compatible SA requirements of its different users and to support SA transactions between team members. Supporting compatible SA involves adopting a ‘right information, at the right time, to the right team member’ design philosophy. This involves the provision of role-based interfaces, displays and tools that are designed based solely on each user’s distinct SA requirements. Displays and interfaces should present only the SA information that is required by each user and users should not be inundated with information required by other agents but not themselves, nor should they have access to tools and functionality that they do not explicitly require. Whilst this means that more sophisticated systems may be required (i.e. customisable interfaces, role-based systems) our case study findings suggest that adopting this approach will support DSA more effectively. Moving onto designing to support SA transactions, the exchange of awareness between agents, our research suggests that designers need an in-depth understanding of exactly what it is that users need to know, when they need to know and what they need to know it for. This allows designers to create information presentation systems (e.g. displays, interfaces) that present SA-related information logically in conjunction with the information that it is likely to be used with. Mapping information together on displays and interfaces supports the SA transactions that need to be made. For example, a land warfare mission support system (analogous to the one analysed in the case study) could present new incoming information regarding a destroyed combat vehicle to combat service support staff (whose job it is to remove and deal with casualties, repair damage and replenish forces) in conjunction with information relating to routes to and from the vehicle, casualty evacuation routes, distances and projected times, combat effectiveness, medical support information (e.g. nearest hospitals etc), force replenishment requirements, repair requirements and also resource availability. In this way, the system is supporting the integration of information elements and thus the SA transactions regarding the broken down
Distributed situation awareness
279
vehicle. On the other hand, the same system could present the information regarding the destroyed combat vehicle very differently in light of different user needs. For example, when presenting the information to the Chief of Staff (who is ‘running’ the battle at the ops table), associated information presented could include the proximal units and their capabilities, the Commander’s effects schematic, the task ORG (resource allocation chart) and the combat service support staff ’s assessment. This information would then support the Chief of Staff in allocating the destroyed units tasks to another unit on the battlefield. Encouraging the design of systems that support the distinct SA requirements of different users and the exchange of awareness between users seems an obvious proposition, and others in the field have articulated the need to consider distinct user SA requirements (e.g. Bolstad, Riley, Jones & Endsley, 2002; Gorman et al., 2006); however, our experiences suggest that it is a principle that is not often followed. Rather, systems appear to be designed based on an understanding of what information they need to present to users, but without an understanding of who requires what information or how the information will ultimately be used by different elements of a collaborative system. As a corollary of this, compatible SA requirements are not supported and the grouping of information on interfaces and displays can be inappropriate. Supporting compatible SA and SA transactions therefore depends on designers having a detailed understanding of the DSA requirements of a system’s different users. Matthews, Strater & Endsley (2004) point out that knowing what the SA requirements are for a given domain provides designers with a basis to develop optimal system designs to maximize human performance rather than overloading workers and degrading their performance. It is therefore recommended that collaborative system design should be driven with a very clear specification of the compatible and transactive SA requirements of the different users. It is important to note identifying DSA requirements does not involve merely identifying the different pieces of information that need to be known, rather it involves going further and identifying what it is that needs to be known, how this information is used and what the relationships between the different pieces of information actually are i.e. how they are integrated and used by different users. Identifying the relationships between different pieces of DSA-related information allows designers to group information meaningfully in their end design. Only with this in-depth understanding of the process and the resultant SA requirements of different team members can compatible and transactive SA truly be supported through system design. With increasing technological capabilities, there is great potential for enhancing DSA in defence systems through the provision of digitised mission support systems; however this is accompanied also by a very real opportunity to create defence systems that can hinder, rather than support DSA development and maintenance. Key issues to pursue relating to the concept of DSA include what information should be presented, in what manner and to which elements of the warfare system, how information can be presented in a more timely fashion and how the accuracy of information presented by command and control systems can be enhanced and ensured. It is hoped that the DSA model and propositional network approach presented in this paper be used to further our knowledge of how defence systems should be designed to better support DSA.
280
P.M. Salmon et al.
References Anderson, J. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press. Artman, H. & Garbis, C. (1998). Situation awareness as distributed cognition. In T. Green, L. Bannon, C. Warren, Buckley (Eds.) Cognition and cooperation. Proceedings of 9th Conference of Cognitive Ergonomics. (pp. 151–156). Limerick: Ireland. Bolstad, C.A., Riley, J.M., Jones, D.G., & Endsley, M.R. (2002). Using goal directed task analysis with Army brigade officer teams. In Proceedings of the 46th Annual Meeting of the Human Factors and Ergonomics Society, Baltimore, MD, pp. 472–476. Endsley, M.R. (1995a). Towards a theory of Situation Awareness in Dynamic Systems. Human Factors, Vol. 37, pp. 32–64. Endsley, M.R. (1995b). Measurement of Situation Awareness in Dynamic Systems. Human Factors, Vol. 37, pp. 65–84. Endsley, M.R., & Robertson, M.M. (2000). Situation awareness in aircraft maintenance teams. International Journal of Industrial Ergonomics, Vol. 26 pp. 301–25. Fiore, S.M., Salas, E., Cuevas, H.M., & Bowers, C.A. (2003) Distributed coordination space: toward a theory of distributed team process and performance. Theoretical Issues in Ergonomics Science, 4, pp. 340 –364. Gorman, J.C., Cooke, N., & Winner, J.L. (2006). Measuring team situation awareness in decentralized command and control environments. Ergonomics, Vol 49, pp. 1312–1326. Hutchins, E. (1995). Cognition in the wild. MIT Press, Cambridge Massachusetts. Matthews, M.D., Strater, L.D., & Endsley, M.R. (2004). Situation awareness requirements for infantry platoon leaders. Military Psychology, 16, pp. 149–161. Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. Freeman, San Francisco. Salmon, P., Stanton, N., Walker, G., & Green, D. (2006). Situation awareness measurement: A review of applicability for C4i environments. Journal of Applied Ergonomics, 37, pp. 225–238. Salmon, P.M, Stanton, N.A., Walker, G.H., Baber, C., Jenkins, D.P. & McMaster, R. (2008). What really is going on? Review of situation awareness models for individuals and teams. Theoretical Issues in Ergonomics Science, Vol 9, pp. 297–323. Shadbolt, N.R., & Burton, M. (1995). Knowledge elicitation: A systemic approach. In J.R. Wilson and E.N. Corlett (Eds.) Evaluation of human work: A Practical Ergonomics Methodology, pp. 406–440. Shu, Y., & Furuta, K. (2005). An inference method of team situation awareness based on mutual awareness. Cognition Technology & Work, 7, pp. 272–287. Smith, K., & Hancock, P.A. (1995). Situation awareness is adaptive, externally directed consciousness. Human Factors, 37, pp. 137–148.
Distributed situation awareness
281
Stanton, N.A., Salmon, P.M., Walker, G.H., Jenkins, D.P. (2009). Genotype and phenotype schema and their role in distributed situation awareness in collaborative systems. Theoretical Issues in Ergonomics Science. 10(1), pp. 43–68. Stanton, N.A., Stewart, R., Harris, D., Houghton, R.J., Baber, C., McMaster, R., Salmon, P.M., Hoyle, G., Walker, G.H., Young, M.S., Linsell, M., Dymott, R., & Green, D., 2006, Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology. Ergonomics, Vol 49, pp 1288–1311.
HUMAN FACTORS MODELS OF MINI UNMANNED AERIAL SYSTEMS IN NETWORK-ENABLED CAPABILITY C. Baber, M. Grandt† & R.J. Houghton EECE, The University of Birmingham, Birmingham, B15 2TT, UK † FGAN Research Institute for Communication, Information Processing & Ergonomics Human-System Integration, Wachtberg, Germany In this paper we explore the potential uses of human factors modelling techniques (descriptive, predictive and process models) in a range of activities involving mini-UAS (Unmanned Aerial vehicle Systems). The primary interest lies in defining the information and decision chains that relate to the use of intelligence from these assets, and the impact that configuring these chains in different ways might have on overall performance. Our interest lies in contrasting different configurations of command structures to see how workload, situation awareness and decision making can be affected, and how intelligence from UASs can be incorporated into Network-Enabled systems.
Introduction The work in this paper is concerned with models of mini unmanned aerial vehicle (UAV) systems. Mini-UAVs have been designed to be operated by two people (in comparison with teams of 8 to 80 in medium and large UAVs) and are being used for a wide range of intelligence gathering operations in the military and, increasingly, by Civil Uniformed Services. While we can accept that a two person team can launch, fly and monitor such a UAV, it is likely that this team will form part of a larger organisation that will receive the resulting intelligence and act accordingly. This organisation will provide support for the UAV team (in the form of Force Protection), or authorise the mission that is being undertaken (in the form of Tasking Orders). Thus, there is a need to consider the relationship between the operators of mini-UAVs and the broader organisational framework within which they will operate.
Unmanned Aerial Vehicles If one views military operations as ‘Find – Fixate – Strike’, then we can consider contemporary warfare as a shift in emphasis in these terms. For example, if the enemy is hidden then the goal of Find becomes far more challenging; if the enemy is operating in an environment with other people living and working nearby, then the goal of Fixate becomes not simply a matter of defining a location but also of providing confirmation of a legitimate target and the goal of Strike becomes one which 282
Human factors models of mini unmanned aerial systems
Table 1.
283
Comparison of UAVs.
UAV
ALT [ft] Crew Craft GCS
Intel
MQ-9 Predator Reaper RQ-2 Pioneer (US Navy) RQ-2 Pioneer (US Marine) RQ Luna RQ-7 Shadow RQ-11 Desert Hawk RQ Aladin
>20 k
80+
4
Pilot PO 2 × PO
Launch and Maintenance recovery crew crew
<15 k
9
8
EP
<15 k
8
5
IP PO SC
<15k
9
6
PO
<15k
22
4
<10k
2
8
<1 k
2
1
IP PO Pilot PO PO Sys Op Sys Op
PO
Launch
Maintain
Airframe Avionics Intel EP EW repair Imagery Engine PO Launch and Maintenance recovery crew crew PO EP Pilot PO Sys Op
Pilot PO Sys Op
requires high levels of precision. Furthermore, the twin goals of Find and Fixate become dependent not only on situation awareness, but also on broader aspects of situation understanding. From this perspective, the history of the Unmanned Aerial Vehicle (UAV) can be seen as a shift from Strike to Find. Table 1 presents a summary of some contemporary UAVs (these examples are taken from USA, UK and Germany). The selection is made to illustrate differences in numbers of personnel involved in UAV operations. Some of the largest UAVs, such as Watchkeeper, might have a Regiment of some 600 personnel, including several UAV batteries, launch and recovery crew, maintenance and communications engineers etc. As we see from table 1 (‘Crew’), a Predator will typically require around 80 people, whereas the mini-UAVs (as we have mentioned above) are intended to be deployed by 2 or 3 person crews. In the UAV column, the US designation of UAV types is used, i.e, ‘R’ refers to reconnaissance, ‘Q’ refers to an Unmanned Aircraft System, and ‘M’ refers to multi-role (the numbers refer to the generation). In the Altitude column, the figures refer to the operating height (typically, heights of 15,000 feet or under involve Line of Sight control of the aircraft). The ‘Craft’ column indicates the number of UAVs that are typically carried by the UAV section.The GCS column refers to the Ground Control System and the crew involved (PO refers to Payload Operator, whose role is to operate the sensors). The Intel column refers to the personnel involved in the interpretation of the data from the sensors, e.g., Intelligence or Imagery Analysts, or the Leader of a small crew. The Launch column illustrates the differences between very large UAVs, which require a dedicated launch crew, to the medium-size UAVs which involve a separate External Pilot (EP) for take-off (who then hands over to the Pilot), to the mini-UAVs which can be launched by the two person crew.
284
C. Baber, M. Grandt & R.J. Houghton
Generic UAS functions A UAV could be used for a range of purposes. However, the predominant use of the mini-UAV can be described as providing situation awareness to Commanders, typically at Section level. This represents some obvious differences between the larger UAVs and the mini-UAVs. First, the intelligence produced from the mini-UAV is for immediate, local consumption rather than distribution through a Headquarters. Second, the mission of the mini-UAV is likely to be planned on a short time-scale, with limited consultation with other agencies. Third, the decision to deploy the UAV can be made by the Section leader. Each of these differences represents a potential shift in Doctrine away from the way in which UAVs are operated. In effect, the mini-UAVs are deployed in a manner similar to the recce patrol, i.e., an immediate intelligence need is defined (a need related to current operations), a tasking order is produced, and a mission initiated. Nehme et al. (2007) provide a taxonomy of UAV missions, that cover (at the top-level) Intelligence/Reconnaissance, Drones, Transport, Extraction, Insertion, Communication, Surveillance. This mixes types of mission with phases of the mission. In this paper, our concern is only with the phases of a mission. We propose that these are: Planning, Insertion, Preparation, Launch, Flight and Intelligence Collection, Termination. Each phase has a number of possible tasks associated with it, and each task could be performed by one or more roles. The roles could be associated with the UAV itself, e.g., Pilot to operate the UAV, Payload Operator to monitor and control sensor data, to Section roles, e.g., force protection, mission planning and operations, to Battle Group roles, e.g., Intelligence analysis, Mission Command. This provides an initial perspective on possible system configuration (in terms of crewing). The distribution of functions implies that some of the tasks are unique to specific individuals and situations, but that many of the tasks can involve several people. One challenge for defining a ‘system’ is to decide whether all of the individuals need to be involved, and whether it is necessary for all individuals to take part in a discussion, or whether it is sufficient to pass a plan from the originator through a defined chain for authorisation. When considering the mini-UAV system, there is a tendency to focus on the 2 or 3 person deployment of the equipment, i.e., in terms of launch, flying and sensor operation. However, this crew size is only feasible when launching (and recovering) from a secure environment, say inside the perimeter of a protected area. When a launch is required in hostile territory, then some Force Protection is required. This might mean an additional 2 or 3 person team being deployed to provide cover, guard the area, provide Force Protection, or might mean a request for air (or other) support. Additionally, the intelligence produced by the UAV crew needs to be passed on to the authority who originated the Tasking Order. This might be a Section or Platoon Commander, which raises a question as to how best to communicate the intelligence. For example, a Platoon was under mortar fire and the Commander decided to deploy a UAV to fly in the direction of the firing (as indicated by sensor data passed via JADOCS). The UAV provided a live video of the mortar team and provided further indication of their location for subsequent strike activity. Thus, the feed from the UAV was used as part of a set of sensor data by the Commander and
Human factors models of mini unmanned aerial systems
285
his team. However, at the Section level, it might not be appropriate to stream live video to personnel who are engaging the enemy. In such circumstances, a succinct verbal précis of the sensor data might be more useful. What is less obvious, is how the intelligence might be recorded and filed for subsequent use. For large UAVs, there are developments of image reference libraries, as well as protocols for the dissemination of sensor data from UAVs. For the mini-UAVs, this is less well developed (presumably as the aircraft are intended from local deployment on immediate missions).
Approaches to modelling Models could focus on the activity of UAV pilots tasks (Martin et al., 1998), or on factors such as situation awareness (Gluck et al., 2007) or workload (Dixon and Wickens, 2003). While there is clearly a need to develop detailed models of how UAVs are operated (to inform both system design and training programmes), this places an emphasis on UAV control as an individual operation. We are concerned with how UAVs become part of a system which will involve several people. In this section, we consider three broad approaches to modelling and their relevance to the discussion of Human Factors of NEC.
Descriptive models Descriptive models provide an outline of the functions and structures of the system under consideration. This is the approach taken in Stanton et al. (2008) and could include Cognitive Work Analysis, Use-case descriptions, or Social Network Analysis. The initial descriptive models (using the WESTT tool developed by the HFI-DTC, Houghton et al., 2008) relate tasks, actors and knowledge. In terms of communication structure, we consider three different structures. We assume personnel in the Headquarters (such as G2, G3, So2(air)), personnel associated with air management (such as ATC, CAOC), personnel associated with the UAV itself (mechanics, pilot, PO, leader, driver), and other personnel associated with ground cover and force protection. By considering which personnel are allocated functions, it is possible to infer who might communicate with whom.
Process models Having developed descriptive models for this mission, we can populate process models to explore how the different configurations affect overall performance. Thus, for example, the system which maps all actors to all functions might have a high communication load than the other systems (which could interfere with task performance). Military processes feature these characteristics as they are often affected by external situations, changes and activities accomplished by more than one operator/soldier etc. Thus, the investigation of information flows and communication in networked systems calls for an adequate representation of this process characteristics in the modeling technique. The modeling technique C3 (Killich et al., 1999)
286
Figure 1.
C. Baber, M. Grandt & R.J. Houghton
Social Network diagram (from left to right): with mapping between actors as one to one; one to many; many to many.
has been developed based on UML to especially meet requirements of process visualization arising from communication, coordination and collaboration in complex socio-technical systems. C3 follows a person-centered approach, i.e. task arising during system operation are assigned to persons involved in the system. In personcentered simulation models person models pull up tasks if the person is in an idle state and its characteristics are compatible to the task requirements. Therefore, people are displayed as so-called swimlanes in the C3 diagram.
Predictive models The process models can be explored dynamically. To date we have used two approaches. The first is based on critical path analysis. This considers both the order in which the tasks are performed and dependencies between these tasks. A dependency could be defined by the actor required to perform the task (assuming that individuals will perform their tasks in sequence) or the information required (assuming that one task will need to be completed before another can start). Figure 4 illustrates the critical path diagram that WESTT produces.
Human factors models of mini unmanned aerial systems
Figure 2.
Extract from a C3-model of a Luna UAV operation.
Figure 3.
Extract from critical path analysis.
287
288
C. Baber, M. Grandt & R.J. Houghton
Figure 4.
Comparison of mission times for different crew sizes.
By running the model using times derived from epochs of the mission, one can compare the alternative crews structures under different operational constraints, e.g., figure 4. After the collaborative modeling process the C3 model is transformed into a simulation model. To allow for this several matrices have to derived from the C3-model describing roles, operational phases and activities. From these matrices a petri-net flowchart (shown in figure 5) can be constructed using commercial software tools for petri-net simulation. These tools allow for a detailed specification of dynamic behaviour by adding own source code to the transitions of the petrinet trajectory. In order to reflect probabilistic behaviour of persons, e.g. varying time demands for task performance, error rates or reaction times, transitions might contain probabilistic transfer models which generate output based on probability density functions. Since the transformation from the C3-model to the executable simulation model is a quite labour intensive procedure ongoing research activities aim for a (at least) semi-autonomous transformation of C3 syntax into object-oriented simulation code (Tackenberg et al., 2007). Performing simulation runs with the probabilistic simulation model results are generated concerning the bandwith of possible system performance with regard to time demands, errors and operator workload in terms of e.g. communication activity. Single simulation runs can be visualized in order to evaluate consequences of different system conditions in the operation.
Discussion The main focus of this paper has been to explore the nature of the mini-UAV system and to illustrate ways in which descriptive and process models can be used to examine alternative configurations. The key issue is that a mini-UAV, while
Human factors models of mini unmanned aerial systems
Figure 5.
289
Extract from a petri-net flowchart with amendments used for specification of trajectory dynamics.
designed for a two or three person team, becomes part of a larger system. On the one hand, it might be possible to specify this system as a completely deployable unit (following the concept of a Combat Air Patrol in US operations). However, this might tie-up personnel in tasks which are not vital for that particular mission. For example, if the UAVs are launched from (and recovered to) a protected compound, then there is less requirement for specific Force Protection personnel; or if the UAV is collecting imagery to be passed onto the Battle Group commander for immediate use, then there might be less requirement for imagery and intelligence analysis. Furthermore, if the UAV is supporting an ongoing contact with enemy, then there might be less of a requirement for passing imagery between the UAV’s GCS and the Section Leader (and it might be more important for the UAV PO to communicate any information verbally). On the basis of this description, we are considering how different configurations for the deployment of UAVs will have
290
C. Baber, M. Grandt & R.J. Houghton
quite different communications requirements (from real-time video to still images to verbal communication).
References Dixon, S.R. and Wickens, C.D., 2003, Control of multiple UAVs: a workload analysis, 12th International Symposium on Aviation Psychology, Dayton, OH Gluck, K.A., Ball, J.T. and Krusmark, M.A., 2007, Cognitive control in a computational model of the Predator pilot, In W.D. Gray (ed) Integrated Models of Cognitive Systems, Oxford: Oxford University Press, 13–28 Houghton, R.J., Baber, C., Cowton, M., Walker, G. and Stanton, N.A., 2008, WESTT (workload, error, situational awareness, time and teamwork): an analytical prototyping system for command and control, Cognition, Technology and Work, 10, 199–207 Killich, S., Luczak, H., Schlick, C., Weissenbach, M., Wiedenmaier, S. and Ziegler, J., 1999, Task modelling for cooperative work. Behavior & Information Technology, Vol. 18, No. 5, 325–338 Martin, E., Lyon, D.R. and Schreiber, B.T., 1998, Designing synthetic tasks for human factors research. An application to uninhabited air vehicles, Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, Santa Monica, CA: Human Factors and Ergonomics Society, 123–127 Nehme, C., Crandall, J.W. and Cummings, M.L., 2007, An Operator Function Taxonomy for Unmanned Aerial Vehicle Missions, 12th International Command and Control Research and Technology Symposium, Newport, RI, June, 2007 Pioro, M., Duckwitz, S., Mann, C. and Grandt, M., 2008, Modeling of cooperative processes of networked teams in critical situations, XXIX International Congress of Psychology, Berlin, Germany, July 20–25, 2008 Stanton, N.A., Baber, C. and Harris, D., 2008, Models of Command and Control, Avebury: Ashgate Tackenberg, S., Kausch, B., Grandt, M. and Schlick, C., 2007, Approaches of an Integrative Simulation Model for Project Development Processes, The 2007 European Simulation and Modelling Conference (ESM 2007), Ostend, Belgium: EUROSIS-ETI, 57–64
HUMAN FACTORS PAST AND PRESENT W. Ian Hamilton Human Engineering Limited This paper examines the trends that can be seen in the development of human factors practice from its roots in experimental psychology, through the emergence of principles of practice, to present day human factors integration (HFI). This evolution has also changed the profile of skills required of the human factors professional; from those of the research scientist to those of a practitioner and crafts-person. It is argued that the practitioner is not concerned with the development of new knowledge, but with the application of a knowledge base to solve problems.
Introduction It is sensible to consider the historical development of a thing as a basis for understanding its future direction and form. The practice of human factors is not today what it was when it first emerged over 100 years ago. That is hardly surprising, since, as Meister argues (1999, p29), the discipline of human factors is fundamentally concerned with the relationship between humans and technology in a work context, and as technology has changed it has redefined that relationship. At the turn of the 20th century most work was physical and relied upon the skill, strength and stamina of the worker. By the time of WWII, work was still largely manual but machines were in widespread use to enhance human productivity and performance. Considerations of physiology gave way to considerations of psychology, particularly the role of skilled behaviour. Taken at its most basic the shift has been from a physical relationship, emphasising human anatomy, biomechanical and physiological properties, to a cognitive relationship that emphasises human information processing capacity and reliability. This has continued throughout the technological revolution brought about by electronics and computerised automation during the latter part of the 20th century. At the same time the work relationship is placed within a human organisation that has been transformed by technology, managerial and socio-economic trends (Adrian Furnham, 2005, p61.) In this 60th year of the Ergonomics Society it is appropriate to look back at the history and evolution of the practice of human factors. In doing so it will be argued that we can see the trends outlined above, but more importantly, we can recognise important paradigm shifts in the methods of practice. In other words, it is not the simple changes in focus due to technology that are important, but the changes in method that have been demanded by new technological and commercial influences. It will be argued that there have been three generations of human factors practice, 291
292
W. Ian Hamilton
each characterised by the dominance of particular methods of application. Today’s practitioners need to be aware of these shifts as a basis for learning and responding to new challenges. The argument summarised here is intended to be a compass as well as a guide to the areas of knowledge that are pertinent to contemporary practice.
1st generation human factors (c1900 to 1950s) What might now in retrospect be called first generation human factors (circa 1900 to the 1950s) had its roots in the scientific management movement of Frederick W Taylor. His seminal work on the productivity of workers at the Bethlehem Steel Works showed for the first time using measurement how productivity was strongly influenced by the design of work equipment, most famously the shovels. These ideas were adopted into the time and motion study methods of the scientific management movement (Taylor, 1911) This work was continued in the building industry by Frank and Lillian Gilbreth (1921) and they had many successes including achieving almost a 300% improvement in productivity in bricklaying. They coined the term ‘therblig’ as a unit of work, or more accurately a system of notation for use in analysing work (see Ferguson, 2000.) It could perhaps be argued that the basis of all forms of task analysis has its roots here. But they are probably best remembered for bringing the discipline of time and motion study to popular recognition in the film Cheaper by the Dozen, which featured a character based on Lillian Gilbreth as a ‘do it all Mom’ with a PhD in Psychology who ran her household of a dozen children in accordance with the principles of scientific management. An excellent summary of this history can be found in Noyes (2001, chapter 1). Skipping ahead to the period between WWI and WWII, we can see a steady rise in the work done in relation to human performance in high-demand situations. Initially this was in response to the US automotive and aviation industries’ need for greater understanding of the demands that their machines would place on human skills, both in production and in operation (Mesiter, 1999, p 146–182). But with the onset of the arms race that preceded WWII, humans were being placed at the controls of increasingly complex machines and being asked to operate them in dangerous and forbidding environments. In this era the notion of the human-machine interface emerged as a specific topic of study for applied experimental psychology. The control display relationship of the human-machine interface was investigated using empirical studies. In fact the distinction between applied psychology and human factors was then somewhat blurred. Nevertheless this work laid an important foundation of stimulus-response psychology that continues today as a useful science-base of human performance capabilities. A notable example is the work by Paul Fitts (and various colleagues, but notably Fitts and Jones, 1947) on the best configuration of controls and displays in cockpits. The dominant methodological themes in first generation human factors were experiment and empirical measurement. Test rigs and prototypes were used to study the human-machine relationship in a range of configurations and simulated operating conditions. A good example of this in Britain was the important work
Human factors past and present
293
by Sir Frederic Bartlett using a simulator of the Spitfire cockpit. Through this approach the interface of many of the contemporary work machines was honed; along with aircraft the list includes radar, sonar, and a range of weapon systems. The limiting factor in this approach was that the design had to have reached a certain maturity of development for the creation of a suitable simulator. But that was a manageable limitation as the pace of change in technology was not fast and most new designs were a re-configuration of the existing components. Consequently the experimental rig was probably also the leading edge of the design work, and the human factors specialists’ reliance on the experimental method was a good fit with the other engineers’ approach.
2nd generation human factors (c1960s to 1990) The second generation of human factors (circa 1960s to 1990) saw the paradigm shift towards less reliance on in-situ experimentation and more problem solving from first principles. This was a necessary response to the increasing specialisation in engineering and the fact that complex systems were no longer ‘designed in one room.’ The human factors specialists were probably somewhat remote from the technological innovation and engineering that were driving the design. The solution was to evolve the systems approach to human factors. A leading exponent of this approach was Alphonse Chapanis who co-wrote the first real human factors text (Chapanis, Garner & Morgan, 1949). He argued that the basic and underlying concept was to recognise that there are five elements to system design: • • • • •
Personnel selection, Personnel training, Machine design, Job design, and Environmental design.
Chapanis and his contemporaries recognised that these components all interact in very complex ways. This was a vital insight and one that played an important part in ensuring that human factors played a role in the major technological and engineering events of the era. In particular it facilitated the integration of human factors expertise and specialists into the NASA manned space flight programme (Chapanis, 1996). The concept has an enduring legacy and is still very influential today in all areas of human factors application. An example is the design of everyday products such as mobile phones and DVD players. It may not seem like personnel training and selection has any relevance to these systems. However, the designer must know the profile of the likely users and their most probable knowledge and expectations and cater for these in the functionality. Similarly the operating steps have to be distilled into an accessible form for presentation in the user manual – this is an incarnation of personnel training. The system concept was adopted in Britain by Chapanis’ contemporaries, most notably Hywel Murell who published the first British text book on human factors and is often credited with coining the term
294
W. Ian Hamilton
ergonomics – although as Noyes (2001, p7) points out it was probably Wjciech Jastrezbowski in 1857 in his writing An outline of Ergonomics, Or the Science of work. The systems approach still relied on the use of the experimental method but also demanded a better knowledge-base of human factors data from which design solutions could be determined, either from reference data or core principles and techniques. The systems approach ensured the integration of human factors into complex systems development projects. But new technologies and unprecedented operating environments meant that iterative design based on experimentation alone was insufficient. Human factors specialists needed to be able to express the human transfer function in a way that allowed them to work seamlessly with their engineering colleagues. The stimulus-response language of experimental psychology was rapidly replaced by the language of information theory and cybernetics. A new focus on information processing channels and bandwidth was used to express human reaction times and decision-making behaviour (Wickens and Hollands, 1999, p4). This new blend of engineering and psychology was, not surprisingly, called engineering psychology. It gave rise to useful models of human-in-the-loop control behaviour as well as equations for reaction time, movement time/accuracy and decision-making time (see Card, Moran and Newell, 1983 for a useful summary.) The best known example is perhaps Fitts’s Law: Movement time to a target = C · Log2 (A/W + 0.5) Where: C is a movement time constant A is the distance to move, and W is the diameter of the target. Fitts’s Law (Fitts and Posner, 1967) and other similar equations continue to have utility today. But more importantly the engineering psychology concept has given rise to a range of valuable theories, models and tools for the predictive analysis of human behaviour in systems. These include signal detection theory, attention theory and models of manual control skill (see Wickens and Hollands, 1999 for a good summary.) There were also important methodological innovations in this era, the prime example being task analysis (see Stammers & Shepherd, 1990 for a good summary.) As mentioned earlier, task analysis can be said to have its roots in scientific management but the basic idea of recording and subsequently analysing human actions has seen many different forms. Some were developed with the objective of understanding skill and prescribing training solutions, others were intended for analysing performance and influencing equipment design. There are many tailored task analysis techniques and a smaller number of generic methods. Most notable is perhaps HTA (Annett and Duncan, 1967). Through the 1960s and 70s task analysis became established as a core technique for the decomposition of the human-machine interaction, and as a primary tool for the human factors specialist. The descriptions it yielded were used to guide the development of training and interface solutions that were often determined from the newly formed knowledge-base of data and design principles.
Human factors past and present
295
Also in this era, the widespread use of computers prompted a major focus on usability and HCI. At first, software interface design followed the same kind of experimental testing that had been the core methodology of human factors since WWII. However, the fast pace of software development meant that there was often simply no time for the prototyping-testing iterative approach. There was then a major push to produce design style guides and models to predict the users’ reaction and performance with new software applications. This further consolidated the methodological shift away from experiment and towards prescriptive design. The concept of user-centred design was firmly established, experimental testing to some extent was overtaken by the idea of user consultation and numerous Standards emerged that emphasised this approach (for example ISO 13407: 1999). All of this reinforced the autonomy and maturity of the human factors discipline, and did much to extend the reach of the profession. Task analysis, user centred design, and engineering psychology’s legacy of information theory models of human performance, along with the accumulated data from the scientific study of work, provided a solid backbone of application techniques and data for human factors and played a significant role in the emergence of human factors as what Meister (1999, p24–26) refers to as a distinctive discipline. However, the discipline concerns were still very much segregated into specialties such as anthropometry, biomechanics, physiology, psycho-physics, HMI, performance (engineering) psychology, social/organisational psychology, statistics, etc. And despite Chapanis’ strong lead there was no real integration of these concerns in a way that supported the ideal he expressed.
3rd generation human factors (c1990s to the present) We are now in the third generation of human factors practice. The systems concept has now crystallised into what has become known as human factors integration. Hal Booher (1990 and 2003) updated and re-launched Chapanis’ notion of the integrated concerns of human factors as MANPRINT and subsequently Human Factors Integration (HFI). The timing was undoubtedly right and the idea took hold. Much detail has been published on HFI but, as for many great ideas, it has at its heart a simple notion; that the concerns of the human factors professional are all related, and that they must be thoroughly integrated with the concerns of the technologists and designers. HFI brings together the various concerns of human factors within the domains of manpower, personnel, training, human engineering, system safety, and health and safety. It supports the consideration of how their interaction will affect the operation of human-in-the-loop systems. HFI examines the trade-off between solutions for equipment design, task design, and the social and organisational management of groups or teams. But just as the integration of human factors concerns is important, so is the integration of these concerns with engineering practices to ensure the delivery of systems that can be operated and maintained successfully. HFI is also a good fit with the formalised processes for systems engineering and this relationship is thoroughly explored in Part II of Booher (2003). Any system
296
W. Ian Hamilton
passes through key stages of development to achieve certain maturity milestones. This can be very complicated and vary depending on the system, but fundamentally the process looks like the left-hand side of the table below. The right hand side of the table illustrates the key HFI focus or themes that are relevant at the maturity stages. Distilled to its absolute core then, HFI may be said to address three key goals for any system design: • To define a feasible role for people, • To specify usable interfaces to equipment, and • To ensure the reliability of human performance. The mapping isn’t perfect but it serves to illustrate a progression that helps to associate the most appropriate human factors techniques to each stage of the development process. This table could be extended to achieve the mapping of particular human factors techniques to each stage to create an effective remit of human factors actions for maturity stages. This has been done for UK Defence acquisition (see Hamilton, Rowbotham and David, 2005). For example, defining the feasible role for humans will require a study on allocation of functions perhaps supported by some preliminary investigation of workload demands to determine manning levels. Some first generation methods are still routinely applied. In the manufacturing stage of development, verification and validation testing will rely on observational and experimental techniques. But in the earlier stages of design there is much more reliance on task analysis and the application of the knowledge base in support of requirements specification. Also, for human reliability analysis, there is greater reliance on desk-top methods that are compatible with formal and probabilistic safety assessment (Rich, Blanchard and McCloskey 2007). More recently the concept of HFI has evolved into HSI (Human-System Interaction). Where HFI emphasised human factors concerns mainly for the development of systems, HSI extends this consideration to operational, and in particular safety management (Swallom, Lindberg and Smith-Jackson, 2003). In so doing it emphasises the role of human factors as an explicit element of system support and safety management. This introduces a new challenge for human factors professionals. Just as we had to learn how to work with our engineering colleagues, we must
Table 1.
System lifecycle and HFI key themes.
System lifecycle or maturity stages
Human factors’ key themes
Concept definition Feasibility Engineering (concept exploration) Preliminary Engineering Detailed Engineering Manufacture/Fabrication Commissioning Operation Decommissioning
Defining a feasible role for humans Specifying requirements for usability Providing assurance of the sustainability of human performance Supporting safety management and an effective safety culture.
Human factors past and present
297
now learn how to coordinate our activities with operational management and safety professionals.
Summary & conclusions This is not a history, but an attempt to document the paradigm shifts in human factors. The purpose is to illustrate the evolution of the philosophy of practice of the maturing and distinctive discipline that we see today. There have been many influential thinkers who have contributed to this progression and I am guilty in this short paper of not having paid homage to enough of them. The reader is directed to the excellent collection of major writings in ergonomics compiled by Neville Moray (2005). Three distinct generations of human factors practice have been described:
1st generation (c1900s to 1950s) This was the era of empirical science applied to work. There was no background knowledge base of data and no application techniques so application relied on observation and experimental testing. There was virtually no distinction between research and application.
2nd generation (c1960s to 1990) Both knowledge and practice techniques became established and human factors emerged as a distinct discipline; still with a heavy reliance on the experimental method, but now with first principles to guide application. The knowledge base is so broad that specialties emerge but there is no overall organizing methodology to support their expression as a coherent model of practice.
3rd generation (1990 to present) HFI/HSI provide an organising framework for the range of human factors concerns and their integration into engineering and safety management processes. This offers a coherent model of practice and serves to introduce some consistency and consensus about the techniques that are used and the timing and criteria for their use. The maturity of the discipline is such that the notion of good practice can now be recognized and discussed. The challenge for the future is to recognise third generation maturity and to consolidate its value and benefits. This will necessitate changes in the teaching and professional development philosophy for the discipline. Two important recommendations are summarised below.
The need to distinguish between research and practice Those working in human factors today can be seen as belonging to one of two main groups (at least for the purposes of general representation.) These are research
298
W. Ian Hamilton
scientists and practitioners. Research scientists can most readily be thought of as scientists belonging to one of the core and underpinning sciences that provide the human factors discipline with vital knowledge resources. These scientists work on applied research topics to do with the human-work relationship in ways that are hardly different from the methods used in first and second generation human factors. According to informal data supplied by the UK Ergonomics Society, they represent about one third of the population of Society members. Practitioners are the remainder of that population. These are people who hardly ever do research work that contributes to the body of knowledge about the humanwork relationship. However, they rely upon and apply that knowledge to the design of work equipment, products, computer software, control systems, tasks, environments, processes, jobs and organisations with the aim of enhancing safety and performance. These are the people whose day to day activities most closely align with the methods of third generation human factors. They are not scientists, although their work is undoubtedly and necessarily scientific. They are crafts-people who take established knowledge and apply it to new situations to create novel solutions. They are to the research scientist as the engineer is to the physicist, and they must both exist in a happy symbiosis wherein each needs and supports the other.
Recognition of core skills for fontemporary practice Today’s human factors professional needs to be equipped with an understanding of the current HFI paradigm. Undoubtedly, HFI as a practice methodology will continue to be revised and improved. It may even be superseded by a better idea. But the underlying paradigmatic features are unlikely to change. Human factors is a distinctive discipline with its own unique knowledge base and repertoire of practice techniques. The practitioner must understand and be able to access these knowledge resources. But more importantly they must be able to affect the craft style of their work and be able to integrate their concerns and objectives with those of their colleagues in engineering, safety and management. In short they have to provide answers.
References Annett, J. And Duncan, K.D. (1967) Task Analysis and Training Design. Occupational Psychology, 41, p 211–221. Booher, H.R. (2003) Handbook of Human Systems Integration, Wiley. Booher, H.R. (editor) (1990) MANPRINT An approach to Systems Integration, Van Nostrand Reinhold BS EN ISO 13407: 1999, Human Centred Design Processes for Interactive Systems Card, S.K., Moran, T. P. and Newell, A. (1983)The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates. Chapanis, A. (1996) Human Factors in Systems Engineering, John Wiley & Sons. Chapanis, A. Garner, W.R. Morgan, C.T. and Sanford F.H. (1949) Applied Experimental Psychology, New York, Wiley.
Human factors past and present
299
Ferguson, D. (2000) Therbligs: The Keys to Simplifying Work, http://gilbreth network.tripod.com/therbligs.html Fitts, P.M. and Jones, R.E. (1947) Psychological Aspects of Instrument Display. Analysis of 270 “pilot error” experiences in reading and interpreting aircraft instruements. Report TSEAA-694-1-12A Dayton Ohio Aeromedical Laboratory, Air Material Command. Fitts, P.M. and Posner, M.I. (1967) Human Performance, Brooks Cole Publishing Company. Furnham, A. (2005) The Psychology of Behaviour at Work: The Individual in the Organisation, Psychology Press. Gilbreth, F.B. and Gilbreth, L. M.(1921) First Steps in Finding the One Best Way to do Work, Paper presented at the Annual Meeting of the American Society of Engineers, New York. Hamilton, W.I., Rowbotham, H. and David R. (2005) Generic and Integrated Human Factors Guidance for Air Systems Acquisition, People & Systems Conference, London. Mesiter, D. (1999) The History of Human Factors and Ergonomics, Lawrence Erlbaum Associates. Moray, N. (editor) ( 2005) Ergonomics Major Writings. Volume 1: The History and Scope of Human Factors. Taylor and Francis. Noyes, J. (2001) Designing for Humans, Psychology Press. Rich, K.J.N.C., Blanchard, H., and McCloskey, J., 2007. The use of Goal Structure Notation as a method for ensuring that human factors are represented in a safety case, in Proceedings of the Second Institution of the Engineering and Technology International Conference on System Safety 2007. Stammers, R.B. and Shepherd, A. (1990) Task Analysis. Chapter 6 of Wilson, J.R. and Corlett, E. N. Evaluation of Human Work. A practical ergonomics methodology. 2nd edition. Taylor & Francis. Swallom, D.W., Lindberg, R.M. and Smith –Jackson, T.L. System Safety Principles and Methods. Chapter 14 of Booher (2003). Taylor, F. W. (1911) Principles of Scientific Management, New York, Harper. Wickens, C.D. and Hollands, J.G. (1999) Engineering Psychology and Human Performance, 3rd Edition, Prentice Hall.
ERGONOMICS CHALLENGES FOR DIGITISATION OF MISSION PLANNING SYSTEMS N.A. Stanton, G.H. Walker, D.P. Jenkins & P.M. Salmon HFI-DTC, School of Civil Engineering and the Environment, University of Southampton, Highfield, Southampton, SO17 1BJ, UK This paper aims to consider the conventional, analogue, mission planning process with the objective of identifying the Ergonomics challenges for digitisation. Prototypes of digital mission planning systems are beginning to be devised and demonstrated, but there has been concern expressed over the design of such systems which fail to understand and incorporate the human aspects of socio-technical systems design. Previous research has identified many of the potential pitfalls of failing to take Ergonomics considerations into account as well as the multiplicity of constraints acting on the planners and planning process. An analysis of mission planning in a Battle Group is presented, based on an observational study by the authors. This study illustrates the efficiency of an analogue process which has evolved over many generations to form the Combat Estimate, a process that is mirrored by forces throughout the world. The challenges for digitisation include ensuring that the mission planning process remains easy and involving, preserving the public nature of the products, encouraging the collaboration and cooperation of the planners, and maintaining the flexibility, adaptability and speed of the analogue planning process. It is argued that digitisation should not become an additional constraint on mission planning.
Introduction to mission planning Mission failure is often thought to be the result of poor mission planning (Levchuk et al., 2002), which places considerable demands on the planners and the planning process. This observation is further confounded by the two general principles of warfare. The first principle is that of the ‘fog of war’ (i.e., the many uncertainties about the true nature of the environment – Clausewitz, 1832) and second the principle that ‘no battle plan survives contact with the enemy’ (i.e., no matter how thorough the planning is, the enemy is unlikely to be compliant and may act in unpredictable ways – von Moltke, undated). These three tenets (i.e., the effects of uncertainty, the enemy and failure on mission planning) require the planning process to be robust, auditable and flexible. Mission planning has to be a continuous, iterative and adaptable process, optimising mission goals, resources and constraints (Levchuck, 2002). Roth et al. (2006) argue that the defining characteristic of command and control is the continual adaptation to a changing environment. 300
Ergonomics challenges for digitisation of mission planning systems
301
Constant change in the goals, priorities, scale of operations, information sources and systems being used means that the planning systems need to be extremely adaptable to cope with these changes. According to Klein and Miller (1999) there are many constraints acting on mission planning, including scarcity of resources, time pressure, uncertainty of information, availability of expertise, and the structure of the tasks to be undertaken. Mission planning requires knowledge of the domain, objects in the domain and their relationships as well as the constraints acting on the domain, the objects and their relations (Kieweit et al., 2005). Klein and Miller (1999) also note that the planning cycles can range from a couple of hours to a few days depending upon the complexity of the situation and the time available. Given all of the constraints acting on the planning process and the need for the plan to be continually revised and modified in the light of the enemy actions and changing situation, Klein and Miller (1999) argue that ‘simpler plans might allow better implementation and easier modification’ ( p. 219). This point is reinforced by Riley et al. (2006) who assert that ‘Plans need to be simple, modifiable, flexible, and developed so that they are quickly and easily understood’ ( p. 1143). Mission planning is an essential and integral part of battle management. Although there are some differences within and between the armed services (and the coalition forces) in the way they go about mission planning, there are also some generally accepted aspects that all plans need to assess. These invariants include: enemy strength, activity and assumed intentions, the goals of the mission, analysis of the constraints in the environment, the intent of the commander, developing courses of action, choosing a course of action, identifying resources requirements, synchronising the assets and actions, and identifying control measures. A summary of the planning process for the US land military may be found in Riley et al. (2006) and the Canadian land forces may be found in Prefontaine (2002). Their description has much in common with land-based planning in the British Army, which is described in The Combat Estimate booklet (CAST, 2007).
Observation of mission planning in a Battle Group The mission planning process has been observed by the authors at the Land Warfare Centre at Warminster and on training exercises in Germany. The observations at Warminster have been both as participant-observers and as normal observers. This section describes the observed activities in the planning process following a Warning Order received from Brigade. For the purpose of this analysis, only the conventional materials (whiteboards, maps, overlays, paper, flipcharts and staff officers’ notebooks) will be examined. The planning is undertaken in a ‘public’ environment when various people contribute and all can view the products. This ‘public’ nature of the products is particularly useful at the briefings, which encourages collaboration and cooperation. It also helps to focus the planners’ minds on the important issues and the command intent. The following vignette describes how a Battle Group Head Quarters was observed to conduct itself in the planning process. Whilst other Battle Group Head Quarters might vary, the basic themes are likely to be similar in processes they follow and products they produce.
302
N.A. Stanton et al.
Warning Order from Brigade arrived The Warning Order (WO) arrived and was handed to the Chief of Staff (CoS) who read the whole document first, highlighting relevant material for himself and the Company level.
Chief of Staff (CoS) creates Company Warning Order The Warning Order was too detailed for Company level, so some editing by CoS was necessary, as well as the inclusion of some additional material to clarify the anticipated task requirements.
Send Warning Order to companies The modified and edited warning order was then sent to the companies below the Battle Group, so that they would have advanced notice of the intention of the orders when they arrived. This gives them an opportunity to prepare in advance of the actual orders.
Create planning timeline The CoS created a planning timeline for the production of a plan to defeat an assault team that had parachuted into their area. There were two hours available to construct the plan (from 1300 to 1500), which meant approximately 17 minutes per question (of the Combat Estimate’s 7 questions). The planning timeline was drawn on a flipchart. The Combat Estimate is a planning process that has been developed over decades and is described in more detail in The Combat Estimate book issued by the Command and Staff Trainer organisation at the Land Warfare Centre in Warminster, UK (CAST, 2007). The Combat Estimate has 7 main questions to guide planners through the process. The activities that were observed in answering each of these questions are presented.
Question 1. What is the enemy doing and why? Question 1 was undertaken by the Engineer and the Intelligence Officer in parallel with question 2. Key terrain features were marked on a transparent overlay placed on top of a map (such as slow-go areas like forests and rivers), as were the approximate disposition of the enemy forces and likely locations (using translucent stickers with the standard military symbols on them from APP-6A – Military Symbols for Land Based Systems), potential avenues of approach, and likely Courses of Action. In this case, it was thought that the enemy assault force was likely to try and meet up with the main armoured forces approaching from the West. The enemy had landed in an area surrounded by forest which gave them some protection, although it was thought that they had not landed where they intended.
Question 2. What have I been told to do and why? The CoS interpreted the orders from Brigade together with the Battle Group commander to complete the Mission Analysis. Each line of the orders was read and the specified and implied tasks were deduced. These were written by hand onto a whiteboard. The Commander’s Critical Information Requirements (CCIRs – which are
Ergonomics challenges for digitisation of mission planning systems
303
linked to the Decision Points in Questions 4 and 5) and Information Requests (IRs) were identified and noted for each task, when appropriate. When the CCIRs/IRs had been derived, the CoS read them off the Mission Analysis whiteboard (expanding where necessary to improve intelligibility) to a clerk who typed them directly onto the Requests For Information (RFI) sheet. The requests were radioed up to Brigade and the responses were tracked on the whiteboard.
Q3. What effects do I want to have on the enemy? The Commanding Officer then drew his required effects onto a flipchart. Three effects were placed above the planning line (SCREEN, CLEAR and DEFEAT) and four effects were placed below the planning line (SCREEN, DEFEAT, GUARD and DEFEND). The two SCREEN effects were placed to prevent the enemy from the West coming to the aid of the group who were being attacked. The CLEAR effect was intended to remove any enemy from the forest, if they were there. The DEFEAT effect was intended to incapacitate the enemy.
Q4. Where can I best accomplish each action/effect? The CoS and Battle Group commander worked on three Courses of Action (COAs) to achieve the Commander’s effects. This was a very quick way to propose and compare three potential COAs in response to the Commanding Officer’s Effects Schematic (remembering that the planning timeline only allowed 17 minutes for each of the seven questions of the Combat Estimate). Meanwhile the Engineer took the Commanding Officer’s Effects Schematic and put the Effects onto the ground, using a TALC (TALC is believed to come from Talc Mica [a crystalline mineral which can be used as a glass substitute] in the military it refers to a clear plastic sheet on which map overlays are drawn) on a paper map. Each Effect became either a Named Area of Interest (NAI) or a Target Area of Interest (TAI). Decision Points (DP) were placed between NAIs and TAIs. The resultant overlay is called the Decision Support Overlay (DSO). It is worth noting that it took approximately 15 minutes to construct the DSO on the TALC (by the Engineer). Between the end of question four (Where can I best accomplish each action an effect?) and the start of question five (What resources do I need to accomplish each action and effect?) the Combat Estimate process was interrupted by the return of the Commanding Officer, who requested a briefing. The CO reviewed the COAs and made some revisions, which was followed by a briefing by the CoS to all cells in the HQ.
Q5. What resources do I need to accomplish each action/effect? The Engineer then constructed the Decision Support Overlay Matrix (DSOM) on paper, taking the NAIs, TAIs and DPs from the paper map and linking them to each other, their location and purpose, and the asset that would be used to achieve the effect. There is a clear link between the NAIs, TAIs and on the hand-written flipchart. The manual production of the DSOM on the paper flipchart offers a process of checking the logic of the DSO, making sure that the NAIs, TAIs and DPs link together and that the assets are being deployed to best effect (i.e., relating each asset to a purpose as the columns are next to each other in the flipchart version of the DSOM).
304
N.A. Stanton et al.
Q6. When and where do the actions take place in relation to each other? The CoS led the discussion of how the force elements would move together through the battle (through a mixture of forward recce (reconnaissance), mounted and dismounted troops, and armoured vehicles) with logistical support and coordinated indirect fire ahead of them (controlled by the fire control lines – see Q7). This was enacted on the map from the start position to the end position to capture the synchronisation issues, which were recorded onto the Coordination Measures whiteboard. The coordination measures were used as a precursor to the construction of the synchronisation matrix.
Q7. What control measures do I need to impose? The fire control measures were developed by the Battle Group Commander, to ensure that the indirect fire ordinance would not be placed on the advancing force elements, or beyond the boundaries of the Battle Group’s area. Five fire control lines were drawn onto an overlay on the paper map and numbered one to five. Each line was given a name, which was entered into the staff officer’s notebook against the number used on the overlay. The convention of naming phase lines was to ensure coordination between the force elements and indirect fire during the operational phase. These activities form the battle plan for the Battle Group. This plan is turned into orders for each of the Companys the Battle Group is directing. Any minor changes to the plan, such as a delay in the timings or a reallocation of a unit (which might become apparent from the updated CCIRs), will mean that a Fragmented Order (FRAGO) is issued to the relevant Company.
Analysis of observed vignette and comparison of media used The record of the media used in this vignette is presented in Table 1, which indicates a variety of media including paper, maps, overlays, whiteboards, flipcharts and staff notebooks. Observation of the planning process suggests that the Combat Estimate method, media and products work well together. The plan was constructed within the two hour time frame, with only 17 minutes per question, and the staff officers had no difficulty using the conventional media. No difficulties were noted working between media, such as taking the effects schematic (question 3) from the flipchart and COA (question 4) from the flip chart to produce the DSO (question 5) on an overlay. Similarly there were no problems noted for taking the DSO (question 5) from the overlay to produce the DSOM (question 6) on a flipchart. The point here is that translation between the media was straight-forward, as all media and products were available for the staff officers to use at all times. The planning media and methods were not seen as a constraint on the planning process. The optimal choice of type and mode of communication is likely to be heavily dependent on the activity conducted. For some activities a textual document or a graphical image is more appropriate than a spoken alternative or vice-versa. The stage of any activity is also likely to heavily influence the optimal communication approach. Table 2 shows the degree of collaboration and cooperation required for
Ergonomics challenges for digitisation of mission planning systems
Table 1. Media/Products
305
Media used during the planning process.
Paper
Maps/ Overlays
Whiteboard
Flipchart
Staff Notebook
Warning Order Planning timeline Q1. BAE/TI Q2. Mission Analysis Q2. CCIRs/RFI Q3. Effects Schematic Q4. COA Q4. DSO Q5. DSOM Q6. Wargame Q6. Co-ordination Q7. Fire control
Table 2. Team work required for each stage of the planning process. Digital MP/BM Estimate Question
Task work or team work
Q1. What is the enemy doing and why? Q2. What have I been told to do and why?
Cooperative activity around the table Isolated intellectual activity followed by collaborative activity around the table Isolated intellectual activity followed by cooperative activity around the table Collaborative activity in which the products are shared
Q3. What effects do I want to have on the enemy? Q4. Where can I best accomplish each action/effect? Q5. What resources do I need to accomplish each action/effect? Q6. Where and when do the actions take place in relation to each other? Q7. What control measures do I need to impose?
different stages of the planning process. There is a clear divide; the latter stages of the process (Questions four to seven) are best supported by collaboration (actors working individually with shared information). The earlier stages are much better suited to cooperative activity where the actors work together on one single product. Walker et al. (2009 – in press) report on the basic human communication structures seen inside a BG HQ, identifying eight key functions, some of which
306
N.A. Stanton et al.
are comprised of further sub-functions. The eight key functions include the Higher Command Formation, the Commanding Officer (CO), Chief of Staff (COS/2IC), the ‘Principal’ Planning Staff such as the IO/G2 (to varying extents it also requires the participation of individual roles such as Recce/ISTAR, Eng, A2/Log and Arty/AD. There are also other ancillary command staff (such as those responsible for more general tasks and information management), what are referred to as sub-units (who are typically carrying out activities live in the battlespace) and, finally, the collection of graphics and planning aids derived from the Combat Estimate (artefacts that represent and transform information in some manner). Walker et al. (2009 – in press) describe the human network as dynamic with different functional nodes and links becoming active under different activity stereotypes. The activity stereotypes that they identified were: providing direction (i.e., the Commanding Officer directing communications and information outwards to subordinate staff in a prescribed and tightly coupled manner); reviewing (i.e., the planning/principal staff communicate in a more collaborative manner with mutual exchange of information and ad-hoc usage of planning materials and outputs); and semi-autonomous working (i.e., the headquarters staff are working individually on assigned tasks and become relatively loosely coupled in terms of communication. The communication channels remain open but used in an ad-hoc, un-prescribed manner. These basic structures account for most of the formal communications. The human network structure is complex, but some of the links are identified in figure 1. As table 2 and figure 1 indicate, the process of mission planning is a collaborative and co-operative process, both in terms of the contribution to the products and the verbal interactions. It is also very obvious that the planning team surrounds themselves with the planning artefacts. Maps, overlays, white-boards, and flipcharts literally adorn every surface. The plan is literally constructed in the space between these artefacts, as information is collected, transformed and integrated from the cognitive artefacts and the interactions between the planning team. The training that planners undergo reinforces the fact that the information needs to be ‘public’, for all to see and interact with. The planning process appears to focus on identifying the constraints (such as the mission, the enemy, the environment, the resources and assets) to help define the possible courses of action. The process also requires an understanding of enemy doctrine and tactics to anticipate their likely behaviour and responses as well as military experience to know what effects are likely to achieve the desired outcome. Although it is difficult to quantify, there is certainly the opportunity for creativity in the way in which the plan is constructed. The planning team are continually trying to identify ways in which they can get the most from their finite resources and assets as well as preventing the enemy from anticipating their strategy. The planning process is also required to be flexible, as the planning process is continuous – as the process of issuing FRAGOs suggests.
Implications for digitisation Moves have been made to develop digital system to support the planning processes (see Riley et al., 2006 and Roth et al., 2006 for just two examples). The focus of
Ergonomics challenges for digitisation of mission planning systems
Figure 1.
307
Relationships between the cells in Battle Group Head Quarters during mission planning.
these activities has been on the products of the planning process for distribution between the planning team and to other people in the network. The challenge to system designers has been to preserve the collaborative, public and creative parts of the planning process as well as supporting different levels of plan fidelity (which will depend on the time available to develop the plan). Perhaps the biggest challenge is to decide what needs to be digitised and what form this digitisation should take. Given that military planning teams have invested considerable effort in developing and refining their planning skills using the traditional media, it would seem appropriate to try and support these activities rather than requiring them to develop a new set of skills. The planning process has evolved over centuries of refinement and improvement (Clausewitz, 1832). Roth et al., argues that much insight may be gleaned from studying the work-arounds and home-grown cognitive artefacts that are being used by command and control teams (such as the so-called ‘cheatsheets’ and sticky notes). The traditional analogue planning process (as described earlier) is certainly abundant with potential metaphors, such as overlays, stickies, routes, COAs and so on. It is worth considering if the conventional media could be captured digitally (by camera, scanner, or other means) if they need to be transmitted as electronic documents with orders or reports, or for wider distribution. As a general design principle, the production of electronic documents should be at least as easy as the production of their analogue equivalents. Baxter (2005) is wary of
308
N.A. Stanton et al.
the inexorable trend to digitise and concerned by the history of technology failing to deliver expected benefits, this is not just linked to military experience (Stanton and Marsden, 1996; Sinclair, 2007). Baxter argues that very few people understand the interrelated issues for technology, operations and human factors (being conversant in just one of these topics is not sufficient). Transformational approaches are likely to cause more problems than they solve. There are concerns that digitisation will lead to additional ‘emergent’ work (Kuper & Giurelli, 2007), both in terms of increasing the amount of ‘direct’ work required as well as the work associated with operation of the digital tools. The emergent nature of the task-artefact cycle has been described by Carroll (2000). Certainly it will not be possible to predict all the ways in which any future system would be used, so it is important to make the system as flexible as possible so that users may adapt it to suit their purposes (Roth et al„ 2006). Kiewiet et al. (2005) noticed that there are marked differences in the planners’ domain knowledge, pointing out that group planning ensures an integrated approach rather than an overemphasis on one planner’s area of strength. The social aspect of planning has not been lost on other researchers (Houghton et al., 2006; Stanton et al., 2006; Walker et al., 2006; Jenkins et al., 2008). The collaborative aspects of planning seem to be a key to successful mission planning. As in the observational case study reported in this paper, Riley et al. (2006) identified different cells contributed to the planning process, such as intelligence, operations, logistics, fire support, engineering and air defence. Kuper & Giurelli (2007) argue that design of collaborative tools to support command and control teams is one of the keys to effective team work. The case study presented by Riley et al. (2006) shows how Human Factors can contribute to the design of a mission planning system which is based on a thorough understanding of the planning process, the demands and constraints. In design of their prototype tools they stress the need to provide a quick visualisation of the plan and the current situation. This enables the current operational picture to be compared with the plans, which may require changes to the plan as the situation changes (Stanton et al., 2008; Stanton, Baber & Harris, 2008).
References Baxter, R. (2005) Ned Ludd encounters Network Enabled Capability. RUSI Defence Systems, (Spring 2005), 34–36. CAST (2007) The Combat Estimate. Land Warfare Centre: Warminster. (Unpublished MoD document). Carroll, J. M. (2000) Making Use: Scenario-Based Design of Human-Computer Interactions. MIT Press: Mass. Clausewitz, C. Von (1832) On War (found at: http://www.clausewitz.com/ CWZHOME/VomKriege2/ONWARTOC2.HTML accessed on: 15August 2007). Houghton, R. J., Baber, C., McMaster, R., Stanton , N. A., Salmon, P., Stewart, R. & Walker, G. (2006) Command and control in emergency services operations: A social network analysis. Ergonomics 49 (12–13), 1204–1225. Jenkins, D. P., Stanton, N. A., Walker, G. H., Salmon, P. M. & Young, M. S. (2008). Using Cognitive Work Analysis to explore activity allocation within military domains, Ergonomics, 51 (6), 798–815.
Ergonomics challenges for digitisation of mission planning systems
309
Kiewiet, D. J., Jorna, R. J. And Wezel, W. V. (2005) Planners and their cognitive maps: an analysis of domain representation using multi-dimensional scaling. Applied Ergonomics, 36, 695–708. Klein, G. and Miller, T. E. (1999) Distributed planning teams. International Journal of Cognitive Ergonomics, 3 (3), 203–222. Kuper, S. R. and Giurelli, B. L. (2007) Custom work aids for distributed command and control teams: a key to enabling highly effective teams. The International C2 Journal, 1 (2), 25–42. Levchuk, G. M., Levchuk, Y. N., Luo, J., Pattipati, K. R. and Kleinman, D. L. (2002) Normative design of organisations – Part I: Mission Planning. IEEE Transactions on Systems, Man and Cybernetics – Part A: Systems and Humans, 32 (3), 346–359. Moltke, H. K. B. G. von (undated) http://en.wikipedia.org/wiki/Helmuth_von_ Moltke_the_Elder (accessed on 15 August 2007). Prefontaine, R. (2002) The administrative estimate in the operation planning process: a toll not well understood. The Army Doctrine and Training System, 5 (4), 4–12. Riley, J. M., Endsley, M. R., Bolstad, C. A. and Cuevas, H. M. (2006) Collaborative planning and situation awareness in Army command and control. Ergonomics, 49 (12–13), 1139–1153. Roth et al. (2006) Evolvable work-centred support systems for command and control. Ergonomics, 49 (7), 688–705. Sinclair, M. A. (2007) Ergonomics issues in future systems. Ergonomics, 50 (12), 1957–1986. Stanton, N. A. & Marsden, P. (1996) From fly-by-wire to drive-by-wire: safety implications of automation in vehicles. Safety Science, 24 (1) 35–49. Stanton, N. A., Baber, C. & Harris, D. (2008) Modelling Command and Control: Event Analysis of Systemic Teamwork. Ashgate: Aldershot. Stanton, N. A., Baber, C., Walker, G. H., Houghton, R. J., McMaster, R., Stewart, R., Harris, D., Jenkins, D. P., Young, M. S., Salmon, P. M. (2008) Development of a generic activities model of command and control, Cognition, Technology and Work, 10 (3) 209–220. Stanton, N. A., Stewart, R., Harris, D., Houghton, R. J., Baber, C., McMaster, R., Salmon, P., Hoyle. G., Walker, G., Young. M. S., Linsell, M., Dymott, R. and Green, D. (2006). Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology. Ergonomics 49 (12–13), 1288–1311. Walker, G. H., Gibson, H., Stanton , N. A., Baber, C., Salmon, P. and Green, D. (2006) Event Analysis of Systemic Teamwork (EAST): A novel integration of ergonomics methods to analyse C4i activity, Ergonomics 49 (12–13), 1345–1369. Walker, G. H., Stanton, N. A., Salmon, P., Jenkins, D., Stewart, R. & Wells, L. (2009) Using an integrated methods approach to analyse the emergent properties of military command and control. Applied Ergonomics (in press).
HUMAN-CENTERED DESIGN FOCUS IN SYSTEMS ENGINEERING ANALYSIS: HUMAN VIEW METHODOLOGY R.J. Smillie1 & H.A.H. Handley2 1
Space and Naval Warfare Systems Command, San Diego, California USA 2 Pacific Science & Engineering Group, San Diego, California USA Human Factors Integration (HFI) broadens ergonomics and focuses on the analyses and tradeoffs among the various HFI domains. HFI, as part of systems engineering, is critical in successful delivery of complex systems. Systems engineers use architecture frameworks as a systematic method for representing complex systems. Architecture frameworks fail to capture the human-centered design aspects needed to ensure the total system effectiveness. By adding a Human View to the other architecture framework views, it can be used to provide an understanding of the human role in systems/enterprise architectures. The objective of this current analysis was to explore the dynamic aspects of the Human View as an effective methodology for HFI practitioners coordinating and collaborating with systems engineers. Data from a system development effort was used to build various Human Views. Modeling and simulation were used to analyze the dynamic operator elements of the system and to augment the Human View process. The analysis results demonstrated that Human View data, when combined with modeling and simulation tools, can be used to assess design impacts and provide systems engineers the architecture information needed for complex system design, development, testing, and production.
Introduction Ergonomics brings the human focus to systems engineering. As an engineering discipline, ergonomics integrates with other engineering disciplines to provide a complete systems approach to the design, development, and evaluation of a given product. Human Factors Integration (HFI), known as Human Systems Integration (HSI) in the United States, broadens ergonomics and focuses on the analyses and tradeoffs among the various HFI domains. According to MOD (2005), HFI is a systematic process for identifying, tracking and resolving human related issues ensuring a balanced development of both technological and human aspects of capability. Bruseberg (2008) states that HFI is an integral component of systems engineering, both of which, must be accomplished together to ensure system success. Systems engineers use architecture frameworks to describe complex systems. An architecture is the structure of components, their relationships, and the principles 310
Human-centered design focus in systems engineering analysis
311
and guidelines governing their design and evolution over time (IEEE STD 610.12, 1995). An architecture framework defines a common approach for development, presentation, and integration of architecture descriptions. It provides the guidance and rules for developing, representing, and understanding architectures based on a common set of products and product descriptions. The application of a framework enables systems engineers and HFI analysts to contribute more effectively to building interoperable systems. Architecture frameworks provide a mechanism for understanding and managing complexity. As currently described frameworks fail to capture the human-centered design aspects necessary to ensure the effectiveness of human operated and maintained systems. The objective of this current analysis was to explore the dynamic aspects of the Human View as an effective methodology for HFI practitioners coordinating and collaborating with systems engineers. Data from a system development effort was used to build various Human Views. Modeling and simulation were used to analyze the dynamic operator elements of the system and to augment the Human View process.
Systems engineering According to the International Council on Systems Engineering (2008), systems engineering is an interdisciplinary approach and a means to enable the realization of successful systems. It focuses on defining customer needs and required functionality early in the development cycle, documenting requirements, then proceeding with design synthesis and system validation while considering the complete problem in terms of operations, performance, test, manufacturing, cost and schedule, training and support, and disposal. Integrated approaches are not new to the ergonomist or the HFI practitioner. In particular, task analysis is one of the fundamental techniques for understanding how integrated systems need to be designed and what role humans play in that system. According to Stanton et al. (2005), “task analysis helps the analyst to understand and represent human and system performance in a particular task or scenario”. And, while Diaper and Stanton (2004) state that there are in the neighborhood of 100 types of task analysis methods, there are many good source documents that provide the appropriate detail (e.g., Annett and Stanton, 2000; Diaper and Stanton, 2004; Stanton et al., 2005). From a practical standpoint, in the design, development, and production of large complex systems, the HFI practitioner has to ensure that the task analysis results are communicated in a language that the systems engineer understands. An architecture framework provides that communication medium.
Architecture frameworks Architecture frameworks provide system engineers a structured language with a common set of products. Frameworks evolved in the support of the acquisition process of complex, military systems. Both the U.K. Ministry of Defence and the U.S. Department of Defense defines different perspectives or views that logically combine to describe system architectures. Architecture frameworks use these views
312
R.J. Smillie & H.A.H. Handley
to breakdown a complex system and to categorize their products. In both the US and UK, the following conventions are used (DoD, 2007): – All View - Describes the Scope and Context (Vocabulary) of the Architecture – Operational View - Identifies What Needs to be Accomplished – Systems/Services View - Relates Systems, Services, Characteristics to Operational Needs – Technical Views - Prescribes Standards and Conventions Each of the four views depicts certain architecture attributes. Some attributes bridge two views and provide integrity, coherence, and consistency to architecture descriptions. To date, however, these conventions have not focused on the human element.
Emergence of the human view There were early attempts to represent humans in the architecture products by including the role of the human and human activities associated with a system (Hildebrand and Adams, 2002; Handley, 2006). In addition, analytical efforts in both Canada and the United Kingdom have been concerned on how to include human activities in an architecture framework (Baker et al., 2006; Bruseberg, 2008). By including the Human View methodology as part of the architecture framework, it allows the HFI practitioner a mechanism to convey an understanding of the human role in systems/enterprise architectures to the systems engineers, and, provides a basis for decisions by stakeholders. By capturing the necessary decision data in the Human View and integrating this view with the rest of the architecture framework, the improved framework provides a complete set of attributes of the system data. Using the Human View methodology to augment the existing systems engineering architecture views, a preliminary schema of the interaction of the individual Human View components was created. The Human View products capture different categories of data with regards to the human system. The data is captured in tables or graphics that describe both content and relationships. The Human View products each capture a different set of human elements, i.e., task, role, network, etc. However, it is the relationships between the elements that impact the performance of the system. The initial Human View Dynamics schema is shown in Figure 1. The numbering of Figure 1 shows the flow of data through a discrete-event model. An event from the environment triggers a task. The role responsible for the task begins processing it. The role may coordinate with team members for information exchange during processing. The way the task is processed may depend on traits of the actual person fulfilling the role and training completed. Use of a system resource to complete the task is included in the model. Additionally, other constraints, such as human characteristics and health hazards may moderate the performance of the task. Once the task is completed; metrics are used to evaluate the task performance.
Modeling and simulation Discrete event models representing the interactions of humans with system components have previously been designed. Using basic components of decision-makers
Human-centered design focus in systems engineering analysis
Health Hazard Human Characteristics
Environment
Manpower Establishment
Input Variable
1
313
Output Variable
triggers
3 2 coordinate Responsible
impacts
Role
Task
Team
6 utilize
performs Output Variable
5
System Resource
Input Variable
4 Training
Metric
Figure 1.
Initial human view dynamics schema.
(humans), tasks, and platforms (system resources), these models have been configured to explore performance metrics of decision maker coordination and mission completion (Handley et al., 1999). Typical models allow input parameters to be varied, constraints to be relaxed and other variables affecting the human system performance to be explored in order to evaluate the effect on the model outcomes, and by inference, on the human system design. The missing element in this is the dynamic aspect of human in terms of performance variance and potential errors. A true dynamic Human View captures aspects of human system components defined in other architecture views. These are dynamic aspects in the sense that states, configurations, or performance parameters may change over time, or as a result of varying conditions or triggering events. It is what HFI practitioners need to provide to systems engineers to either (a) convey the complete set of human/operator requirements, constraints, and limitations, or (b) assess existing systems designs (architectures) to ensure human/operator requirements, constraints, and limitations are addressed. It can be used to compare other design aspects (when capturing “asis” behavior aspects) and to assess design decisions (by modeling “to-be” behavior). Elements of the dynamic Human View include: – States (e.g., snapshots) and State Changes – e.g., organizational/team structure; function/role assignments to people; team interaction modes; demands on collaboration load (e.g., the need to spend effort in building shared awareness, consensus-finding, communicating); task switches/interruptions. – Conditions (e.g., triggering events or situations; scenarios) – critical/frequent/representative/typical scenarios; operational constraints (e.g., extensive heat stress); time conditions: sequence, duration, concurrency.
314
R.J. Smillie & H.A.H. Handley
– Time Units – timeline; defined mission phases; sequence of consecutive tasks. – Performance measures (observed or predicted) – e.g., workload; decision speed; team interaction/collaboration style; trust in commander’s intent, quality of shared awareness/coordination/implicit communication. (Different methods of assessment, such as observer rubrics, may be necessary for the more subjective attributes). As a case study to support the Human View process, the Improved Performance Research Integration Tool (IMPRINT) was used. While any modelling and simulation tool that has the capabilities to assess dynamic aspects of human behaviour, IMPRINT has a long accredited history. IMPRINT is a stochastic task-network modeling tool designed to help assess the interaction of people and system performance from concept and design through field-testing and system upgrades (Mitchell et al., 2008). An important feature of IMPRINT is that it helps researchers and designers evaluate operator mental workload while testing alternate systemoperator function allocations. Wickens’ Multiple Resource Theory (MRT) is the basis for the IMPRINT workload algorithm (Wickens, 1991). According to MRT, human mental resources for handling tasks are limited. When an individual is required to perform multiple tasks at the same time, he or she is utilizing the same limited resources for the concurrent tasks. This combination of limited cognitive resources and multiple task demands may result in high workload that, in turn, may lead to a greater number of errors, increased task time, or both. Low task demands can lead to a disassociation with the environment, complacency or boredom where the operator is slow or ineffective in completing sporadic tasks. Physiology studies of the human cortex have supported Wickens MRT theory and demonstrated that concurrent tasks using two different resources, visual and auditory, result in reduced performance (Shomstein and Yantis 2004). Simulation models require that the information captured in the discrete Human View products be formatted or entered into the model in such a way that the simulation can use the information to produce its output. For example, the IMPRINT user interface prompts the modeller to enter the “Operators” involved in the simulation. The modeller can go to one of Human View products to retrieve this data. IMPRINT also provides a graphical interface for the modeller to input and link the set of tasks that compose the scenario that will be simulated. The modeller can derive this information from the task dependencies in a different Human View product. IMPRINT then allows the modeller to associate tasks with operators in order to simulate workload values for the operators during the course of the scenario. Simulation models also allow designers and analysts the ability to make hypothetical changes to the system design and analyze the results. Multiple iterations of the simulation can be conducted with variations in the system design to identify the interactions between variables and identify the most effective design. Using the workload simulation of the IMPRINT model, tasks can be reallocated between operators during multiple runs of the simulation to identify the best balance between operator workload and task performance. As is often the case, multiple variables impact the mission performance in conflicting ways. As constraints are also included in the model, such as resource constraints, i.e., limited communication channels or physical resources
Human-centered design focus in systems engineering analysis
Table 1.
315
Step-wise process to create IMPRINT model using human view data.
Step
Imprint model
Human view data
1 2 3 4 5 6 OUTPUTS
Operators Mission Network Diagram Operator Assignment Resource-Interface (RI) Pairs Task Time, Accuracy, and Task Effects Performance Moderators Mission Results Task Performance Operator Workload
Roles Tasks Task-Role Matrix System Interfaces Performance Standards/ Measures Constraints Metrics Metrics Metrics/Constraints
required for the task, and human factor constraints, such as temperature and rest requirements, as well as other factors including training, personnel availability, etc., simple comparisons are no longer sufficient. Multiple simulations must evaluate trade-offs between different constraints in order to achieve the best operating point. The purpose of the dynamic Human View is to capture the interaction of the human system components (Handley and Smillie, 2008). An effective modeling and simulation tool can assess the static Human View data under dynamic situations and provide the system engineer designers with a complete set of HFI criteria.
Method In order to evaluate the validity of this approach, information based on the U.S. Army’s Future Combat System (Mitchell and Brennan, 2008) was used to build various Human View products. The Future Combat System is an advanced systems design approach that focuses on the combat brigade equipping the individual elements as an integrated unit and based upon the environment it will be used. Next, an experimental model was created in IMPRINT from the Human View data. By creating an actual IMPRINT model using Human View data, issues with differences in terminology, level of detail, and content descriptions could be addressed. The process used to create the model is described in Table 1. A subset of the Human View data from the Future Combat System was used to define the input for the IMPRINT model. Operators were defined by the Human View roles. Task descriptions were used to create a network diagram for a specified mission. The task-role combination provided the operator assignments. The performance standards/measures were used to define the expected task times, accuracy, and outcomes. Constraints help determine moderators that impact performance (e.g., heat, etc.). Finally, IMPRINT outputs provide the data that describe the overall success/failure of the mission, task performance completion and potential errors, and operator workload. These results can then be used to support the overall
316
R.J. Smillie & H.A.H. Handley
Human Views supported by IMPRINT example.
Environment
OUTPUT –Manpower Requirements
Health Hazard Input Human Variable Characteristics
Manpower Establishment
Scenario (Network) triggers impacts
Task
Role Dependencies
Task –Role Assignment
Role
responsible utilize
Tasks to Metrics Mapping
Metric
performs System Resource Output Variable
Output Variable
Task – System Interfaces
coordinate
Team
KSA to Training Mapping
Training
Input Variable
OUTPUT – Crew Performance OUTPUT – Crew Workload
Figure 2.
Human View products supported by modeling and simulation tool.
systems engineering process ensuring that the human/operator requirements are satisfied.
Results Once the model was created, a baseline simulation was executed to provide expected levels of mission performance parameters of time and accuracy. IMPRINT provides for the overall development of a network task model that accounts for the tasks and the types and numbers of operators performing those tasks. It also provides the opportunity to examine the effects of unexpected outcomes. After the model was simulated twenty-five times, the outcome measures were analyzed in terms of performance and workload. In this test case, there were no overall mission failures. But, a more detailed analysis did show that some tasks did fail. The simulation results also showed that the workload of the individual operators was well within the boundaries set for the scenario. In addition, IMPRINT allows post-run modifications to assess alternative configurations. For example, different role to task allocations, based on manpower estimates, can result in different workload outcomes for specific roles. Training options can be exercised to evaluate different types of crew members fulfilling roles. System interfaces can be evaluated in relation to task metrics. The IMPRINT dynamic model can help set realistic system requirements and evaluate the impact on operator workload. Figure 2 highlights the Human View products that were supported by IMPRINT.
Human-centered design focus in systems engineering analysis
317
Discussion Several efforts in various countries are underway to define and structure the Human View as a viable methodology for HFI practitioners to coordinate and collaborate with the system engineers on major complex system design, development, test, and production (DNDAF, 2008; Handley and Smillie, 2008; Handley, 2008; Bruseburg, 2008). Progress on these efforts includes the definition, description, and examples of the various types of Human View products that complement the architecture framework views used in systems engineering. While the ergonomists always had a set of tools and processes to support system development (e.g., task analysis, function allocation, etc.), the Human View products facilitate a more structured language for communicating with the other engineering disciplines during system development. The Human View products are derived using an ergonomic approach, namely, a top down method analyzing human gaps in existing architecture frameworks, or based on specific needs that evolved during the course of the architecture development to capture specific human view data. As with ergonomic analyses in general, any top down approach needs to be supplemented with the bottoms up analysis to ensure all aspects of human constraints and limitations are considered. By augmenting the traditional five-stage structured analysis approach used by systems engineers (Wagenhals et al., 2000), HFI practitioners can use the Human View methodology to provide a fully integrated set of products that ensure an effective and efficient design, development, and production process.
Conclusion The analysis results demonstrated that Human View data for a complex system, such as the Future Combat System, can be used to assess design impacts when combined with a simulation tool, such as, IMPRINT. The IMPRINT tool provides the simulation environment, for the dynamic Human View. The dynamic Human View is critical in the architecture framework approach because it captures the dynamic aspects of the human system components defined in other views. States, configurations, or performance parameters may change over time, or as a result of varying conditions or triggering events. The dynamic Human View is the method for communicating enterprise behavior. It is important for capturing “as-is” and assessing design decisions implications of the “to-be”. The dynamic Human View provides the basis for executable models that can be supported by simulation tools. In the current effort, the Human View products provide additional information that was not used by the IMPRINT tool. In addition, the IMPRINT model requires some information that is not captured in the Human View products. A strong alignment between the two environments exists, however, and can be improved by augmenting the overall process, including the use of other human performance modeling and simulation tools.
318
R.J. Smillie & H.A.H. Handley
References Annett, J.and Stanton, N.A. (eds) 2000, Task Analysis, (Taylor and Francis, London) Baker, K., Pogue, C., Pagotto, J., and Greenley, M. 2006, HumanViews: Addressing the Human Element of Capability-Based Decisions, (TTCP HSI Symposium, DSTO, Australia) Bruseberg, A. and Lintern, G. 2007, Human Factors Integration for MoDAF: Needs and Solution Approaches, (Proceedings 17th Annual International Symposium, International Council of Systems Engineering (INCOSE), San Diego) Bruseberg A. 2008, Human Factors Integration (HFI) Support for MODAF: The Development of Human Views, (Unpublished, MOD HFI Defence Technology Center. United Kingdom) Department of Defense 2007, Department of Defense (DoD) Architecture Framework Version 1.5, Department of Defense (DoD) Architecture Framework Working Group, Washington) Department of National Defence and Canadian Forces 2007, Department of National Defence Architecture Framework (DNDAF), (RDIMS Document Number OTT_TUNN-#335862, Ottawa) Diaper, D. and Stanton, N.A. 2004, Handbook of Task Analysis in HumanComputer Interaction, (Lawrence Erlbaum Associates, Mahwah) Handley, H.A.H. and Smillie, R.J. 2008, Architecture Framework Human View: The NATO Approach, Systems Engineering, 11(2) 156–164 Handley, H.A.H., Sorber T., Dunaway J., 2006, Maritime Headquarters with Maritime Operations Center, Concept Based Assessment Focus Area: Organization, Human Systems Performance Assessment Capability, Final Report, (Pacific Science & Engineering Group, San Diego) Handley, H.A.H., Zaidi, Z.R., and Levis, A.H. 1999, The Use of Simulation Models in Simulation Driven Experimentation, Systems Engineering, 2(2) 108–128 Hildebrand G, and Adams C, 2002, Human Views in the Systems Architecture Framework, Report, (Naval Surface Warfare Center, Dahlgren) Institute of Electrical and Electronics Engineers, 1995, IEEE STD 610.12, Standard Glossary of Software Engineering Terminology, (The Institute of Electrical and Electronics Engineers, Inc., Piscataway) International Council of Systems Engineering (INCOSE) 2008, http://www.incose. org/ Mitchell, D. and Brennan, G. 2008, Mission Centered Human System Analysis Using Future Combat System Vignettes, Report, (Army Research Laboratory, Aberdeen) Ministry of Defence, 2005, The Ministry of Defence (MOD) Human Factors Integration (HFI) Process Handbook, (MOD HFI Defence Technology Center, United Kingdom.) Ministry of Defence, 2008, The Human View Handbook for the Ministry of Defence (MOD), (MOD HFI Defence Technology Center, United Kingdom) Stanton, N.A., Salmon, P.M., Walker, G.H., Baber, C. and Jenkins, D.P. 2005, Human Factors Methods: A Practical Guide for Engineering and Design, (Ashgate Publishing Company, Hampshire)
Human-centered design focus in systems engineering analysis
319
Stanton, N.A., Hedge, A., Salas, E., Hendrick, H. and Brookhaus, K. 2005, Handbook of Human Factors and Ergonomics Methods, (Taylor and Francis, London) Shomstein, S. and Yantis, S. 2004, Control of Attention Shifts between Vision and Audition in Human Cortex, The Journal of Neuroscience, 24(47), 10702–10706 Wagenhals, L.W., Shin, I., Kim, D., and Levis, A. H. 2000, C4ISR Architectures: II. A Structured Analysis Approach for Architecture Design, Systems Engineering 3(4) 248–287 Wickens, C. D. 1991, Processing Resources and Attention. In D. Damos (ed) Multiple Task Performance, (Taylor and Francis, London) 3–34
SUBMARINE SAFETY AND SPATIAL AWARENESS: THE SUBSAFE GAMES-BASED TRAINING SYSTEM R.J. Stone1 & A. Caird-Daley2 1
Human Interface Technologies Team, College of Engineering & Physical Sciences, University of Birmingham 2 Department of Systems Engineering and Human Factors, Cranfield University
A feasibility study commissioned in 1998 by Flag Officer Submarines (FOSM) evaluated the human factors issues of adopting new forms of interactive media to enhance the spatial awareness of safety-critical systems onboard Trafalgar Class vessels. The study delivered an early virtual demonstrator that combined simple, realtime 3D models of the main submarine decks and compartments with 360◦ digital image panoramas of complex, cluttered areas. Digital panoramas were chosen to overcome the huge financial and computational costs of reproducing every pipe, cable, valve and subsystem evident onboard vessels of this class in 3D. More recently, however, hardware and software technologies developed for computer games have proved to be more than capable of delivering quite detailed virtual environments on PC platforms and gaming consoles for nonentertainment applications, at a fraction of the cost than was the case 8 years ago. SubSafe is one such example of what can be achieved using gaming technologies, exploiting freely available, freely distributable software. SubSafe is a proof-of-concept demonstrator that presents end users with an interactive three-dimensional model of part of a typical Naval Base quayside and a Trafalgar Class submarine. The virtual submarine model comprises 3 decks forward of bulkhead 59 (aft of the control room), which accounts for some 35 compartments and over 500 different objects (such as valves, pipes, firefighting and life-saving units, important control panels and so on). This paper presents the background to the SubSafe project and current plans for pilot studies conducted in conjunction with the Royal Navy’s Submarine School in Devonport. The studies are investigating knowledge transfer from the classroom to the real submarine (during Week 7 of the students’ “Submarine Qualification Dry”, or SMQ(D) course), together with general usability and interactivity assessments. The paper also briefly considers future extensions into other submarine training domains, including periscope ranging and look-interval assessment skills and the planning and rehearsal of approach and docking procedures for submersible rescue operations.
Introduction In many countries, submarines – be they conventional diesel or nuclear “hunterkiller” boats (SSKs and SSNs) or strategic missile platforms (SSBNs) – help 320
Submarine safety and spatial awareness
321
maintain a level of deterrence in increasingly fragile socio-political situations. Once described in the early 1900s by Vice Admiral Arthur Wilson as “underhand, unfair, and damned un-English”, submarine fleets are, today, routinely deployed in support of a range of duties, from coastal protection and patrol duties to the support of scientific research in some of the most inhospitable places on the planet. Of course, with such a range of hostile natural and conflict-ridden environments, danger is inevitable and, since the year 2000, there have been a number of incidents involving submarines from some of the world’s major sea powers (e.g. see “Major Submarine Incidents Since 2000”, catalogued by Wikipedia (2008)). Two incidents in particular, both involving submarines of British origin, drove the motivation to reassess the way in which present-day submariners are trained, particularly with regard to their spatial knowledge relating to the layout of safetycritical systems and items of life-saving equipment. The first occurred onboard Her Majesty’s Canadian Submarine HMCS Chicoutimi (an ex-UK Upholder Class SSK). On October 5, 2004 the Chicoutimi was running on the surface through heavy seas to the northwest of Ireland. The submarine was struck by a large wave and some 2000 litres of water entered the vessel through open fin hatches. The water caused electrical shorting in the vicinity of the captain’s cabin and the ensuing fire disabled 9 members of the crew as a result of smoke inhalation. Unfortunately one of the crew later died. The second incident occurred onboard HMSTireless (shown in Figure 1) in March 2007. Tireless was taking part in under-ice exercises along with a US submarine north ofAlaska. During what should have been a routine lighting of a Self-Contained Oxygen Generator (SCOG) – part of the survival equipment in the submarine’s Forward Escape Compartment – the generator exploded, killing two crewmembers and seriously injuring a third. The submarine’s crew managed to manoeuvre the vessel to thin ice, some 2 miles away, at which point she was able to surface safely and implement ventilation procedures. There is absolutely no evidence to suggest that deficiencies in current basic submarine qualification (SMQ) training contributed to the outcomes of these incidents. Indeed, in both cases, the actions of the submarine crews were highly praised by the Boards of Inquiry. However, there is anecdotal evidence to suggest that these two incidents have prompted both the Canadian and Royal Navies (and, indeed, the Royal Australian Navy) to review submarine safety training in general, particularly for naval personnel joining vessels for the first time (or after a long period of detachment). Part of these reviews involves an assessment of current forms of classroom-based (or Submarine Qualification (Dry) – SMQ(D)) training media that are in use and the potential benefits offered by more interactive digital systems, including interactive 3D (i3D). This paper addresses one such interactive media review being conducted by the UK’s Human Factors Integration Defence Technology Centre (www.hfidtc.com), in conjunction with instructor and student submariners from the Royal Navy. In particular, the paper describes a pilot study evaluating a prototype i3D product called SubSafe – a “Virtual Reality” (VR) simulator based on contemporary video games technologies and designed to enhance spatial awareness training during SMQ(D) classroom exercises.
322
Figure 1.
R.J. Stone & A. Caird-Daley
HMS Tireless – a British SSN – taking part in an Arctic scientific survey [2007; Source – HMS Tireless Website].
SubSafe Origins Space does not permit a thorough coverage of all related submarine design and operation studies, such as the VESUB submarine handling and piloting system (Seamon et al., 1999) and other UK, US and Australian simulation-based design activities (e.g. Biegel et al., 1998; Edwards & Nilsson, 2000; Stone, 2002). Instead, the paper concentrates on some of the early UK developments in exploiting interactive media for vessel design, together with more recent and relevant developments in i3D spatial awareness training for submarine environments. Working collaboratively with Rolls-Royce and Rolls-Royce & Associates (RR&A), Vickers Shipbuilding & Engineering Limited (VSEL, now part of BAE Systems) began their venture into Virtual Reality in 1993 with the ambitious goal of modelling complete submarine environments. At VSEL in Barrow-in-Furness, a huge one-fifth scale model of an SSBN was housed within a dedicated storage facility (Figure 2, upper image). Model human figures (such as the popular “Action Man” or “GI Joe” dolls) were used for basic ergonomics evaluations. The financial and manpower resources required to build and maintain models of this complexity over the entire life of the Vanguard class of submarine was huge, not to mention the insurance implications of protecting such a facility. VSEL saw the potential of VR to eradicate these physical models in the life cycle of a submarine (Figure 2, lower image), thereby not only bringing major commercial benefits to the company’s design, evaluation and concurrent engineering activities, but also enhancing the project review, evaluation and training requirements of the Royal Navy and Ministry of Defence. Today, BAE Systems at Barrow uses dedicated real-time
Submarine safety and spatial awareness
323
Figure 2. One-Fifth Scale Model of an SSBN Operations Room (Upper Image); SSBN Virtual Machinery Space (Lower Image) [Source – Author (Stone’s) Archives].
visualisation facilities in support of the Astute Class submarine programme and have demonstrated significant time-to-market savings. Also in the mid-1990s, the Human Factors team at RR&A conducted two short studies with the University of Nottingham and VR Solutions Ltd of Salford to evaluate the potential of VR for resolving submarine maintenance engineering problems, thereby reducing time spent within hazardous operational environments. A “desktop” VR demonstrator based on Superscape’s Virtual Reality Toolkit (VRT) simulated submarine spaces and included key landmark features such as ladders, piping, electrical generation machinery and switchboards. Viewpoints could be created that simulated the viewpoints of different sizes of engineers’virtual “bodies” whilst standing, kneeling and lying prone. Additional research addressed the design of a virtual “toolbox”, in order to simplify the process of dismantling, repairing and refitting of equipment. The second RR&A project utilised another VRT model, this time of a submarine reactor plant comprising major landmark features such as vessels, ladders, machinery components and large-bore piping. Underpinning the 3D model were dose-rate contours of the reactor plant, based on information generated from a health physics database. As the end user moved throughout the VR model, accumulated and instantaneous radiation doses were on a simple windowed interface surrounding the 3D view, together with a plan view and virtual coordinates of the user’s position, together with elapsed time (Figure 3). The plan for this system (which was never realised) was to evaluate how such an i3D trainer could optimise task
324
Figure 3.
R.J. Stone & A. Caird-Daley
Interactive 3D Model of Submarine Reactor Plant [Source – Author (Stone’s) Archives].
performance by investigating the effect on dosage level of different access routes, working positions, shielding or plant states. Despite these early developments, it is only very recently that attention has focused on the use of i3D technologies to help the real end users of these designs – the submariners themselves – to become familiar with the layout of vessels before they actually embark on operational duties. In particular, research and development projects in the UK and Australia are under way to address the use of affordable and accessible training products that not only replace legacy training media in the classroom (e.g. whiteboards, posters and PowerPoint presentations), but can also be used by students for revision or rehearsal outside of scheduled training periods. In particular, low-cost hardware and software technologies made available from the computer or video games industry are revolutionising the i3D community (Stone, 2005), not only in defence sectors but also in aerospace, medicine, cultural heritage and education, to mention but a few.
Virtual Reality & Submarine Spatial Awareness (UK; 1997) Following the withdrawal of funding in 1996 for a “Multi-Level Machine-Based Trainer” (MLMBT) project for SSBN (Vanguard Class) submarine familiarisation, a 2-year study was conducted by VR Solutions Ltd with the support of Flag Officer Submarines (FOSM) to assess the future relevance and potential of VR technologies for SMQ(D) training. FOSM’s main concern was the diminishing access on the part of submarine students to key physical assets for the purpose of familiarisation training, evacuation procedures, safety equipment location and incident muster procedures. Such assets need to be at their maximum operational efficiency for obvious defence and commercial reasons. Even when they are available at the dockside, during refit for instance, they are not the most ideal of training environments, courtesy
Submarine safety and spatial awareness
325
Figure 4. “Fading” Digital Panoramas Used to Bring Contextual Detail to a Simplified 3D Compartment Model [Source – Author (Stone’s) Archives].
of the presence of a maintenance workforce and the fact that many compartments will bear no resemblance to the at-sea condition of the submarine. The final study report (Stone & Connell, 1999) described a solution based on combining simple 3D models of the main submarine decks and compartments with digital panoramas (as supported by such interactive 360◦ products as QuickTime VR or Ipix) and a limited proof-of-concept demonstrator was developed. Digital panoramas were chosen to overcome the huge financial and computational costs of implementing every pipe, cable, valve and other object or system evident onboard vessels of this class. By navigating around and through the simple VR representations of internal submarine spaces and landmark features, end users could launch panoramas from specific node points, thereby being able to appreciate the visual complexity of each compartment (Figure 4). “Hot spots” within the panoramas and 3D scenes were linked to additional 3D objects, containing recognisable models of important and safety-critical equipment. In the case of the 3D objects, once the hot spot had been interrogated (via a single mouse click), the object appeared, complete only with those features and labels relevant to its identification and operation. The end user was then able to initiate any animated or interactive features (including sound effects) to demonstrate important procedures and processes. Although these features were only
326
R.J. Stone & A. Caird-Daley
ever implemented within a 3D representation of the Forward Escape Compartment (Figure 4), the demonstrator confirmed the potential of exploiting interactive 3D media for Submarine Qualification, Dry (SMQ(D)) virtual walkthrough training.
Australian Location and Scenario Training System The Location and Scenario Training System (LASTS) was developed by the Royal Australian Navy to evaluate the exploitation of computer games technologies in the training of submariners joining Collins Class submarines. LASTS currently takes the form of a 3D reconstruction of the Collins Class Main Generator Room (MGR) and is powered by the Unreal Tournament games engine. An experimental project designed and conducted by a student at Perth’s Edith Cowan University presented participants with a simplified exercise requiring the location of items relevant to a 12-point Safety Round performed inside the MGR (Garrett, 2007). Five student submariners were exposed to LASTS and then required to conduct the same exercise on-board a Collins class submarine. This mode of learning was compared to traditional non-immersive classroom teaching involving five additional student submariners who were also required to complete the same exercise inside the MGR. A mixture of qualitative and quantitative approaches to data collection and analysis was used to ascertain the effectiveness of LASTS as well as the contributing factors to this and learners’ perception of the value of the environment. The research is ongoing, but preliminary results suggest that training within LASTS was at least, if not more effective than traditional non-immersive training methods. There is also some evidence to suggest that the LASTS students possessed a better overall spatial representation of the MGR compared to those who received traditional classroom based training.
The SubSafe Training i3D Prototype Nearly 7 years after the original FOSM study described above had been completed, the HFI DTC team was approached by representatives of the MoD’s Submarine Integrated Project Team (IPT). Having read the FOSM report and been made aware of other games-based and i3D simulation projects, sufficient interest had been generated to enable the SMQ(D) project to be revisited. In conjunction with a small Shaftesbury-based company, Incredible Box, and training personnel at SMQ (South), part of the Royal Navy’s Submarine School based at HMS Drake in Devonport, the SubSafe i3D prototype was completed in 2007. SubSafe consists of a 3D model of a British Trafalgar Class SSN, moored at a virtual reconstruction of part of HM Naval Base in Devonport (Figure 5). The computer-generated boat can be explored by student submariners in a “first-person” game style – navigating decks and compartments (e.g. Figure 6) simply by using a PC or laptop mouse and the arrow keys of a conventional keyboard. All decks forward of bulkhead 59 (aft of the control room) have been modelled with landmark features in mind, as have key items of safety-critical and life-saving equipment (e.g.
Submarine safety and spatial awareness
327
Figure 5. SubSafe External View of the SubSafe Virtual SSN. [Source – Author (Stone’s) Archives].
Figure 6. SubSafe Internal View of Virtual Operations Room [Source – Author (Stone’s) Archives].
fire extinguishers and hose units, high-pressure air valves, emergency breathing system (EBS) masks, etc.). Whilst it has not been possible to include every valve, pipe and subsystem, the virtual submarine model consists of some 30 compartments and 500 different objects. The software underpinning the SubSafe tool, Ogre3D, can be downloaded from the Web, distributed completely free of charge and can be hosted on a laptop costing less than £500. Although the ability to interact with specific items of equipment throughout the submarine was not a formal requirement imposed on the first version of SubSafe, it was decided to demonstrate “best practice” in virtual interactivity (see also Stone, 2008) by allowing users to select and manipulate a range of objects throughout the 3D submarine model. The interactive style was based on that designed for a virtual Tornado Avionics Training Facility (ATF), delivered to RAF Marham in the late 1990s (Moltenbrey, 2000; Stone, 2004). The ATF was designed to train avionics
328
R.J. Stone & A. Caird-Daley
engineers to isolate (pre-programmed) faults in line replaceable units (LRUs), either by inspecting individual virtual LRUs, or by applying appropriate virtual test equipment at various points in the avionics systems network. In keeping with the ATF system design, objects such as valves, items of safety equipment, hatches, doors and panels can be selected, “extracted” and manipulated using simple “point-and-click” mouse inputs.
SubSafe Evaluation Trials The objective of the SubSafe study described in this paper is to evaluate the effectiveness of this games-based spatial awareness trainer as a supplement to current SMQ(D) classroom-based. Specifically, it is hypothesised that real-time, i3D training afforded by SubSafe will enhance students’ abilities to locate safety-critical items of equipment on board an actual Trafalgar Class submarine in week seven of their SMQ(D) course. The evaluation study also sought to capture the reactions of the student participants and the SMQ Instructors to SubSafe as a training device. Since SubSafe has been designed specifically for the SMQ(D) course, the evaluation protocol was developed in close consultation with SMQ Instructors. Particular consideration was given to the logistics of integrating the study into the SMQ(D) course format to minimise any disruption to training, to limit the number of personnel involved and to make optimum use of the resources required (e.g. submarine access). Ethics approval was also required from the Ministry of Defence Research Ethics Committee (MoDREC) prior to the collection of trial data.
Pilot Study Design An independent-groups experimental design is being employed. There is one independent variable (IV), namely the method of presentation of SubSafe. The IV has three levels (i.e. three different presentational modes) and, as a consequence, three groups. These are: • a control group, members of which are not given access to the SubSafe system; • a “passive presentation” group who receive a video presentation of SubSafe delivered by SMQ Instructors to illustrate the layout of the boat and the location of specific safety critical items of equipment; • a “free roam” group who are afforded hands-on access to SubSafe and are required to navigate the virtual submarine model in real time, undertaking search-andlocate tasks for the same specific items of safety-critical equipment as were presented to the passive presentation group. One dependent variable is being measured: correctness in locating and identifying safety-critical items of equipment whilst undertaking a short walkthrough test on board a Trafalgar Class submarine during week seven of the SMQ(D) course. Participants’ ratings of the effectiveness of SubSafe as a training device are also being captured by means of a post-trial questionnaire. The questionnaire is administered
Submarine safety and spatial awareness
329
to all participants, including the control group (who will be given hands-on access to SubSafe after their week seven walkthrough). Each of the trials in the study are to be replicated over a 6-month period, thereby doubling the number of participants in each group and increasing the explanatory power of the statistical treatment of the data.
Week Seven Walkthrough Test Twenty-six items of submarine equipment have been selected for inclusion in the walkthrough test on the basis that they were safety critical items with which all submariners are required to be familiar prior to joining a boat for the first time. Each item of equipment exists in the form of a 3D object onboard the virtual submarine. For each of the 26 equipment items included in the test, the questions require a verbal response from participants (e.g. ‘how do you operate item x?’), followed by a “seek-and-find” component, requiring each participant to locate the actual item on the submarine. The week seven walkthrough test has been developed in conjunction with the SMQ Instructors. Indeed, the walkthrough test has been designed to be administered by the SMQ Instructors and integrated with their own week seven walkthrough question schedule. Space constraints onboard SSNs means that it is impossible for one of the HFI DTC researchers to accompany an SMQ Instructor without compromising the execution of the walkthrough. A response recording sheet has been prepared to record whether the response/location components of the test were delivered (a) correctly with no prompting, (b) correctly with prompting, or (c) incorrectly (despite prompting). This categorical recording approach was devised to accord with an established protocol of providing limited prompting during week seven walkthrough tests. Finally the trial items were checked against SubSafe to ensure they were accurately rendered and labelled within the synthetic environment.
Participant Reaction Questionnaires An 8-item questionnaire has been developed to gauge the reaction of the student participants to SubSafe as a training device. This questionnaire is administered after the week seven walkthrough test. Two forms of the questionnaire have been developed, one for use with the passive and free-roam groups, and one for use with the control group to accommodate their delayed access to SubSafe. Participants respond using a 5-point rating scale, from “strongly disagree” to “strongly agree”, and they are also asked to provide qualitative responses to support their ratings.
Procedure Prior to participating in the trials, students are given a pre-trial briefing and a detailed participant information sheet. They are also required to read and sign an informed consent form. Participants also complete a short background information questionnaire to capture their age, intended area of specialisation on completion of their SMQ course, and their experience of playing computer or video games.
330
R.J. Stone & A. Caird-Daley
Participants in the passive and free-roam groups are then given access to SubSafe. The participants in the passive group receive SubSafe as a video presentation during week six of their SMQ(D) course. Week 6 is normally reserved for revision activities – the SMQ Instructors felt that SubSafe would be well suited to being presented at this time, with minimal disruption to the rest of the course. Participants in the free-roam group are given individual access to SubSafe, again in week six of the SMQ(D) course. In week seven of the course, participants in all three conditions undertake the walkthrough test onboard an actual Trafalgar Class submarine. The test is administered by SMQ Instructors as they accompany individual student participants around the boat. Following the week seven walkthrough, participants in the control group are given hands-on access to SubSafe, undertaking the same task as was set for the free-roam group. Participants in all conditions then complete the post trial questionnaire. All participants are then debriefed by means of a comprehensive debriefing sheet.
Early Feedback At the time of writing and due to delays in securing MoDREC ethical approval for the SubSafe study, followed by the lack of availability of submarines for Week 7 SMQ Walkthrough evaluations, the experimental evaluation is incomplete (estimated date of completion is early April, 2009). Consequently it has not been possible to report the knowledge transfer of results in this paper. However, as well as the participating SMW trainees, the SubSafe prototype has been demonstrated to a number of Royal Navy personnel and has, without exception, received favourable initial feedback. The system has even been demonstrated to Senior Ratings onboard the SSN HMS Tireless whilst at sea in the English Channel in December 2007. Managing expectations has been problematic in some cases, as younger users expect more of the simulation than it currently delivers. These users have to be reminded regularly that SubSafe is a games-based training application, not a first-person action game! More formalised feedback will be obtained once the evaluation trials have been completed. In addition to the experimental trials, the design and development experiences with the SubSafe project have been collated and used as evidence in support of a recently-published Human Factors Guidelines document for i3D and games-based training (Stone, 2008). This document has been designed to inform Human Factors specialists, simulation and games developers and potential procurers of future interactive 3D systems for education and training. The main message of the document is that, no matter how “good” the pedagogy, the capability of an interactive 3D or games-based learning system to educate and train can be completely destroyed if content, fidelity and interactive technologies are implemented inappropriately and without a sound Human Factors underpinning. The contents draw on case study material from the past 10–12 years, illustrating the application of human-centred design processes in projects as diverse as close-range weapons simulation, surgical skills and knowledge training and explosive ordnance disposal operations.
Submarine safety and spatial awareness
331
The results of the current experimental evaluation of SubSafe are contributing to subsequent editions of these guidelines.
References Biegel, P.E., Brown, S.P., Mason, T.C., & Poland, D.D. (1998). Development of a Personal Computer Simulation-Based Multimedia Ship Control Training Program. Johns Hopkins APL Technical Digest. 19(4), 470–481. Edwards, M., & Nilsson, U. (2000). Virtual Reality & Simulation Based Design – Fact or Fiction in Submarine Platform Design. Proceedings of UDT Pacific 2000. Garrett, M. (2007). Sub Space: Enhancing the Spatial Awareness of Trainee Submariners using 3D Simulation Environments. Unpublished Undergraduate Project Report. Edith Cowan University (Perth, Australia). Moltenbrey, K. (2000). Tornado Watchers. Computer Graphics World. 23(9). Seamon, A.G., Bradley, S.K., Hays, R.T., & Vincenzi, D.A. (1999). VESUB Technology Demonstration: Project Summary. Naval Air Warfare Center Training Systems Division (Orlando) Report No. A565263. Stone, R.J. (2002). Interactive 3D and is Role in Synthetic Environment-Based Equipment Acquisition (SeBA). Report prepared under MoD Contract RD01805103 for the Director of Integrated Systems, Dstl, Farnborough. Stone, R.J. (2004). Rapid Assessment of Tasks and Context (RATaC) for TechnologyBased Training. Proceedings of I/ITSEC 2004 (I/ITSEC; Orlando, Florida), 6–9 December, 2004. Stone, R.J. (2005). Serious Gaming – Virtual Reality’s Saviour? In Proceedings of Virtual Systems & Multi-Media Conference; Ghent, Belgium. 773–786. Stone, R.J. (2008). Human Factors Guidelines for Interactive 3D and Games-Based Training Systems Design. Human Factors Integration Defence Technology Centre Document. Available from www.hfidtc.com. Stone, R.J., & Connell, A.P. (1999). Virtual Reality & SMQ Training. An Initial Investigation and Recommendations. VR Solutions Ltd Internal Report. Wikipedia (2008). Major Submarine Incidents Since 2000. http://en.wikipedia. org/wiki/Major_submarine_incidents_since_2000. Last checked: 20 September, 2008.
CRITERIA FOR ASSESSING SOCIAL AND ORGANISATIONAL IMPACTS OF EMERGING TECHNOLOGIES P. Sirett, R. Braithwaite, R. Cousens, J. Marsh & M. Mason SCS Ltd The study examines the implications of emerging technologies within the context of Network Enabled Capability (NEC) to assess the Social & Organisational (S&O) impact of the introduction of new equipment, a developing Human Factors Integration (HFI) domain. The revolutionary potential of the network is highlighted through a literature review together with the risks of applying technology to existing processes rather than those that may be enabled by the network itself. This paper explains the core of the study, which is the identification of criteria for assessing social and organisational impact, and provides an initial framework for future studies within the S&O theme. The S&O criteria have been derived by reviewing military papers and conducting workshops with military subject matter experts. For validation purposes, a number of emerging military technologies were assessed against the criteria identified. The S&O criteria found to be valid are presented as a series of tables.
Introduction This study was conducted on behalf of the Haldane Spearmen Consortium, as a number of high-level Ministry of Defence (MOD) strategy papers have identified convincing evidence that emerging technologies will have Social and & Organisational (S&O) impact. A detailed understanding of what these S&O impacts might consist of and their subsequent effect on operational capability has remained less clear. This study was conducted with the aim of identifying the potential S&O impact of emerging technologies within military networks by way of identifying specific S&O issues across a number of key areas, such as teams, processes, command and control, culture and linking users to technology. The purpose was to consolidate and set out S&O criteria that can be used in the development of reliable and valid S&O measurement tools. For the purposes of this study emerging technologies were defined as those distinctive technologies that will underpin future technical capabilities. Emerging technologies consist of a broad range of developments from immature technologies in the early proof-of-principle stages through to novel defence applications of existing, mature technology. They will define the future environment in which UK Armed Forces have to operate, can potentially redefine the way in which modern warfare is conducted, and may render existing capability obsolete. 332
Criteria for assessing social and organisational impacts
333
The notion that the S&O structure of an organisation can be influenced by novel technology is not new, and history offers many examples of opportunities that have either been taken or missed. These include the Battle of Crecy in 1346 after which organisations and structures were adapted to optimise the new capability offered by the longbow, a technology that had actually emerged 200 years previously. Conversely the French defeat during WWII in 1940 offers a dramatic example of the penalties associated with clinging to the organisational structures of a previous era. There is always the temptation to ‘make yesterday perfect’ by applying new technology to current Tactics, Techniques, Procedures and overarching organisational structures rather than changing them to accommodate new technology. Networked Enabled Capability (NEC) is a theme that underpins this study as it is the one capability that transcends the military operating environment and defines the potential for revolutionary S&O change. Military NEC is a long term change programme for the integration of sensors, decision-makers, effectors and support capabilities to achieve a more flexible and responsive military. Commanders will be better aware of the evolving military situation and will be able to react to events through voice and data communications. The revolutionary effect of a network is not confined to the defence sector; there are many organisations that have undergone profound S&O change as a consequence of embracing the full potential of a network (e.g. the US supermarket chain Wal-Mart). The appropriate military organisation vis-à-vis networks, groups and NonGovernment Organisations (NGOs) will remain relatively unknown until the situation for deployment develops. That is, the relevant organisations required for deployment will be identified based on a number of variables, such as type of threat, availability of units, or types of NGO required. Moving from traditional hierarchies to networked operations will create S&O impacts that are only just beginning to emerge from recent operational deployments; many continue to remain unknown. For these reasons this study has viewed the organisation as an operational network, regardless of the size and type of units involved. S&O impact should be addressed within the context of HFI. HFI ‘ensure[s] that the human component is adequately considered in capability development.’ It is noted that Social and Organisational is one of seven domains considered within the HFI process. In fact, it is a relatively recent addition and has emerged in response to the organisational complexity of providing NEC. Due to the recent introduction of the S&O HFI domain there are currently no standardised S&O criteria in use for the assessment of systems integration within a military network.
Method Research papers, experiments and conceptual papers were reviewed to identify those S&O aspects that come to bear on networks. Two workshops were held with ex-military personnel who have worked in command posts and within the MOD procurement process. MOD strategy papers were reviewed and discussed at the workshops to identify further criteria that relate to systems integration. The S&O criteria identified, through the literature review and workshops, were then validated
334
P. Sirett et al.
by discussion with a number of Integrated Project Teams (IPTs) that were involved in major equipment procurement programmes with an in-service date of 2013. All the programmes selected for inclusion involved the delivery of new capability rather than replacing a current capability with new equipment (i.e. revolutionary rather than evolutionary, as discussed later in this report). Through these discussions various technologies were assessed against the S&O criteria identified. Those S&O criteria that could be usefully utilised in assessing emerging technologies were kept and are presented in the results section of this paper.
Literature review Disruptive innovation There is an important distinction between ‘sustaining’ and ‘disruptive’ innovation that applies to the potential of the network. In essence, a ‘sustaining’ innovation improves the performance of an established product whereas a ‘disruptive’ innovation can truly flourish only when managed by a non-traditional organisation (Christensen, 1997) that is implemented concurrently with the new technology. Thus, technologies and systems that are viewed as sustaining will have little impact on S&O issues, whereas disruptive technologies will have a greater impact. Disruptive innovations have seen the merging of old and new organisations, an increase in status of some personnel, unit or service, the emergence of new career paths and old career paths becoming redundant. Further, culture change requires the open-mindedness of senior personnel, the linking of elements through simulations and training, doctrinal refinements and force structure modifications ensuring that interoperability issues are considered (Capt Pierce, 2004).
Sense-making and learning S&O issues may be driven not only by the technology but by the complexity of the environment and the manner in which the organisation interprets and understands what is happening. Military organisations can be viewed as ‘interpretation systems’, especially due to the relationship between Intelligence, Surveillance, Target Acquisition, and Reconnaissance (ISTAR) systems and the commander (Turner, 2007). Commanders will therefore seek input from various information sources across the network to inform planning cycles. The social interactions that will take place across the network will be based on the various objectives and needs of units in the network, these being driven by the interpretation of the battlefield. It is therefore important to understand elements of the network in the context of the wider environment, and how groups interact within the network to make sense of complex situations. Making sense of information from a systems perspective means that there will be an input stage, some form of processing, and an output. The Systems approach recognises that feedback from the environment is important (Gibson et al., 1994). However, the traditional Systems approach does not fully account for the complexities of the individual or group of people within the input, processing and
Criteria for assessing social and organisational impacts
335
output mechanisms. It is important to understand and define the physical aspects, the process activities, the epistemological aspects of the system, and indeed the system of systems (Leedom et al., undated). However, at a high level, organisations will have inter-dependencies and so changes in one part will affect other parts of the organisation (Smither, 1994). It is therefore important that any change in system or technology is assessed in terms of the inter-dependencies, the activities, and input and outputs required of staff across the network that relate to receiving information and turning it into knowledge. It is imperative that new technologies enable and facilitate organisational sense-making so that learning occurs between individuals, groups, and distributed and ad hoc teams.
Social learning Social learning occurs within groups as a result of the interaction between the situation and personal factors, such as motivation, attitude and personality (McKenna, 2006). Organisational values that have been found to support social learning include: empowerment, trust, forgiveness, commitment, decision openness and a culture of information sharing. It is important to note that the aspects of trust, decision openness, and an information sharing culture are not necessarily facilitated by networked operations per se (Warne et al., 2004) . A culture of information sharing may not necessarily facilitate co-operation between units and may hinder selfsynchronisation, depending on the roles and status of individuals and units involved.
Roles and status The US experience of early networked operations in the Gulf (1991) highlighted issues with role and status. When adopting a network-centric approach to warfare, the status and power of the US Air Force (USAF) increased above that of the Army and Marines during the first phase of the war: the aerial bombardment. The position and role of USAF commanders within the network enabled them to prioritise attacks on targets from accurate information. However, it has been reported “that USAF commanders appeared to select targets that fitted with their own model of how the war should be fought, adopting to select strategic targets, and tended to ignore Army and Marine nominated targets” (Warne et al., 2004). This example shows that the network does not necessarily facilitate co-operation and understanding even though it can deliver timely and accurate information. Rather, political, individual and group dynamics remain within networked operations to such an extent that the social fabric of the network needs to be better understood. It also appears that the unit’s role, status and location in the network are important factors, and to some extent may even be dictated by the individuals in post. Unit position and role in the network were found to have an impact in a multinational battle experiment that used Unmanned Airborne Vehicles (UAVs) to provide intelligence to a maritime coalition defence force (Crebolder et al., 2007). That is, when the unit’s Commanding Officer (CO) performed the role of Officer in Tactical Command (OTC), crews reported an increase in understanding, Situational Awareness (SA), and the number of interactions with other ships.
336
P. Sirett et al.
Virtual groups As individual and group learning is important to the functioning of a network it is pertinent to investigate the key characteristics of virtual groups and how these characteristics may be influenced by social interactions. Membership of the group will be dependent upon how the network is organised (e.g. star, circle) in combination with the strategic, operational and tactical objectives. However, core characteristics required of virtual groups can be identified. A study investigating the effects of social conformity in a virtual environment (Obden et al., 2006) found that group members were influenced by social interactions during the early part of the experiment. That is, members agreed with other members even when the solution presented to a task problem was incorrect. This effect dissipated over time as members became more confident and realised that disagreeing with others did not have negative consequences. In addition virtual group members needed time for confidence building with other members they had not seen or met previously, and they also required time to assess the reliability of the system. These findings are extremely relevant to the military context due to the fact that there will be fewer face-to-face interactions within future Command and Control (C2) environments as teams will receive orders and distributed intent through the network (e.g. secure voice, Voice Over Internet Protocol (VOIP) and chat rooms). From a systems perspective, it is argued that organisations will form, develop, mature and decline in relation to environmental demands (Gibson et al., 1994). As personnel at each node within the network are part of the total system, the development, maturity and decline of the organisation may be based on how social interactions evolve over the course of operations; with trust and confidence issues being at the fore. The notion of maturity is also related to the formation of teams (i.e. forming, storming, norming and performing), which it has been argued will take place within future HQ and will be based on knowledge acquisition (Mathieson and Dodd, 2004).
Organisational culture Organisational culture has been defined as ‘a specific term to describe the customs, practices, and attitudes of people within the organisation, but only while they are exercising membership of it’ (Kirk, 2004). Organisational culture may be both formal, through known practices advocated through doctrine and Standard Operational Procedures (SOPs), and informal, which are shown through the attitudes that people develop whilst operating within the working system. As military personnel are deployed on operations, they exercise membership of the organisation for longer and more intense periods than is the norm for civilians. Thus, organisational culture is a particularly important aspect to consider when integrating new technologies within military organisations. From the perspective of organisational psychology, culture is defined as: “A pattern of basic assumptions, invented, discovered or developed by a particular group as it learns to cope with its problems of external adaptation and integral integration . . . taught to new members as the way to perceive, think and feel in relation to problems.” (Schein, 1990).
Criteria for assessing social and organisational impacts
337
This definition, taken from mainstream organisational psychology theory, is pertinent to the UK military as it addresses group adaptation, integration and problem solving, which directly relate to the concept of agile and self-synchronising operations. The definition points to the notion that organisational cultural values are passed down as new members join the organisation, and that this process is through social interactions. It is further argued that different occupations show different types of culture that influence people’s behaviour (Schein, 1996). In a military organisation, therefore, there may be a number of sub-cultures, based on unit and branch or even trade values. As the UK military uses new technology and moves towards a fully integrated NEC, a new culture may emerge (e.g. in attitudes and expectations) from networked relations and the use of new technologies. Self-synchronisation is itself a cultural change in thinking. It may be that power and politics, role, and status have a greater or different impact on cultural values than previously seen within a hierarchical structure. The exact nature of what a ‘networked culture’ will and should contain is not well understood at this point due to the changes that are currently taking place with both methods and systems of delivering military force. An important factor that should be taken into account during technology implementation is the expectations and attitudes of personnel who are required to operate new technologies or utilise information derived from new technology, otherwise change may occur slowly or in an unexpected direction (Kirk, 2004). This change may take place in a number of social or organisational ways, which are difficult to predict at this point.
Harmonisation Having personnel who acquire the appropriate skill-sets to work effectively within the network will mean more than having the appropriate culture, training regime, work processes and social learning conditions. At the front end it will be important to promote jobs and associated roles detailing the specific competencies that are required for effective performance. Then it will be necessary to recruit and select people with the aptitude to acquire the appropriate knowledge, skills and attitudes (KSAs) that reflect the idealistic service and unit culture. If the network and associated roles and tasks within the operating environment are matched to the attitudes and expectations of staff (i.e. service and sub-unit cultures) then it is argued here that retention will remain high. In order that the appropriate service and sub-unit cultures are facilitated and maintained, the military will need to adopt valid and robust mechanisms and management tools. It is argued that future warfare will be about creative people management “in order to hold and maintain mechanical elements together, such as, careers, culture, capability and leadership” (Warne et al., 2004). The integration of technology needs to take into account the harmonisation mechanism and processes, which include entry into the organisation and the formal socialisation process (i.e. basic training). The harmonisation process should extend beyond initial socialisation mechanisms and include those aspects that relate to the infrastructure for operations, the capability, and strategic and unit objectives. Should technology implementation change activities and tasks significantly (e.g. greater automation or greater technical skills) and therefore the nature of the role,
338
P. Sirett et al.
it may be that other human-centred issues relating to the infrastructure, such as pay and rewards, will need to be assessed and adjusted accordingly.
Results: Social and organisational criteria tables A number of S&O criteria emerged from the literature review and workshops that need to be considered during technology integration. These S&O criteria are presented in the tables below. The tables highlight key criteria and example questions that need to be asked during technology integration.
Table 1.
Social criteria for gaining cultural values.
Serial
Cultural values
Context
1.
Open mindedness
2.
Mutual trust
3.
Empowerment
4.
Forgiveness
5.
Commitment
6.
Decision openness
7.
Obedience
8.
Contact with local environment
9.
Co-operative
10.
Responsible
11.
Authoritative
Are senior staff committed and willing to ensure the technology is integrated effectively? (e.g. try new ways of working units together) Does the new technology create and facilitate trust between individuals and teams? (e.g., more or less face-to-face communication?) Do the technology and associated leadership and processes empower personnel to make decisions and act? Is the new technology likely to cause a blame culture? (e.g. is the technology transparent?) Does the new technology increase or decrease commitment to the task/job and unit? (e.g. the extent to which individual and unit attitudes and expectations are met, tasks become mundane, tasks are more autonomous) Does the new technology make decisions and actions auditable and transparent? Does the new technology facilitate professional behaviour? (e.g. can law of armed conflict and unit standards be easily subverted?) Is interaction with local nationals inhibited or facilitated by utilising the new technology? (e.g. more remote working) Does the new technology promote co-operative working for inter- and intra-team goals? (e.g. enables collaborative goal setting, units are aware of others actions or intended plans) Does the new technology make personnel/units responsible for decisions and actions? (e.g. is responsibility distributed or focused, is responsibility at lower levels accountable as the situation unfolds?) Is individual and unit authority altered due to technology implementation? [Note: may relate to unit or service status]
Criteria for assessing social and organisational impacts
Table 2.
339
Social criteria for attaining effective synchronisation.
Serial
Effective synchronisation
Context
1.
Common goals
2.
Collaboration
3.
Group learning
4. 5.
Distributed team learning Confidence and trust building
6.
Team development cycle (forming, storming norming and performing)
Are common goals within the network facilitated? How? Is collaboration facilitated in the network? I (e.g. objectives across the network can be met effectively via collaborative planning mechanisms and tools) Is group learning facilitated? (e.g. output is represented in an easily interpretable format, communication is enhanced) Is distributed team learning facilitated? Are confidence and trust facilitated? (e.g. can source be verified quickly, is face–to-face communication possible via a live feed?) Is the team maturity cycle facilitated? (e.g. can distributed teams form and move to performing effectively in a short time frame?)
Table 3.
Social criteria for attaining effective command and control.
Serial
Command and control
Context
1.
Leadership
2.
Balance of direction and delegation
3.
Appropriate control
4.
Defines missions
5.
Provides resources
6.
Clear command intent
Does the nature of leadership change? (e.g. is leadership more distributed or linked to one individual?) Will the technology allow commanders to provide a balance of direction and delegation? (e.g. commanders become more involved with micro management) Does the new technology provide individuals or units with appropriate control? (e.g. access and provision of specific information required by others) Can missions be defined in a clear and concise manner? (e.g. multiple methods of providing key messages) Can resources for others in the network be provided as appropriate? (e.g. not held up due to additional tasks being created in the logistics chain) Can command intent be defined in a clear and concise manner? (e.g. is there any doubt displayed by recipients of the intent, are all elements of intent provided such as key messages etc?)
340
P. Sirett et al.
Table 4.
Organisational criteria for assessing high level infrastructure.
Serial
High level infrastructure
1.
Service role status
2.
Unit role status
3.
Personnel role and status
4.
Doctrine
5.
SOPs
6.
Performance metrics
7.
Effect
8.
Stability of current groups and teams
Table 5.
Context Does technological change affect the role the service plays in delivering force or protection? (e.g. greater or different types of fire power, different effects, enhanced ISTAR activities) Does technological change affect the role of the unit? (e.g. more central to ISTAR activities, more remote to area of operation, greater impact on operational picture) Does technological change affect the status of personnel? (e.g. individuals can push, pull or withhold specific information providing greater control of information) Are doctrinal refinements required to ensure that the technology is effectively and safely used during operations? Do SOPs need adjusting to account for new ways of working with the technology? Will new performance metrics be required for measuring operational effectiveness? Is the effect that is delivered by the group, unit or service altered by the inputs or outputs that can be derived by utilising the new technology? [Note: if the unit role changes so may the effects] Will groups/units need to be created, merged or disbanded in order that the full potential is gained from the technology? (e.g. old units re-trained or new functional cells created)
Organisational criteria for linking components and processes.
Serial
Components and process
1.
Interoperability issues
2.
Simulations
3.
Collective Exercises
4.
Training
Context Is the new technology likely to cause interoperability issues above and beyond ‘plug and play’ requirements? (e.g. integration of additional liaison officers) Are simulations required to understand how new technologies and systems link together with older systems and processes in order to examine the combined effects on performance and behaviour? Are UK collective exercises required to refine, change or validate processes, SOPs, and doctrine? Does unit level training need to be amended to allow for new individual and team skills to be learned? (e.g. technical knowledge, new team processes and SOPs)
Criteria for assessing social and organisational impacts
Table 6.
341
Organisational criteria for coupling users to technology.
Serial
Coupling
Context
1.
Inputs
2.
Processing, activities and tasks
3.
Collaborative process
4.
Output
5.
Interdependencies
6.
Feedback mechanisms
Does the form of input to other nodes in the network facilitate learning? (e.g. voice, text, diagrammatic, dynamic visuals) Have the processing, activities and tasks (e.g. team interactions, individual activities, and automation) altered? (e.g. more reliance on automated solutions) Is the collaborative process facilitated across the network? (e.g. do nodes require additional or different input/output from other nodes or teams?) Does technological change affect the output provided? (e.g. voice, text, diagrammatic, dynamic visuals) Does the change facilitate learning? What interdependencies are affected across the network and are there wider adjustments within the organisation that need to be made? (e.g. inter or intra team processes). Will the input/output affect other nodes in the network especially in terms of sense-making and learning? Has feedback from within the network and wider environment changed? Does feedback enhance learning and sense-making relating to the operational picture? Is the feedback likely to cause problems for any wider support staff? (e.g. live feeds of destruction) [Note: related to inputs and outputs]
Table 7.
Organisational criteria related to harmonisation.
Serial
Harmonisation
Context
1.
Career paths
2.
Marketing
3.
Recruitment
4.
Selection
5.
Basic and advanced training
6.
Management tools
7.
Pay and rewards
Are new career paths required to account for any changes in individual, unit and service roles? Does the marketing of military jobs reflect the activities, tasks and culture of the role? (e.g. is the new technology represented on flyers, pamphlets, and internet etc) Do recruitment activities reflect the tasks and culture of the role? (e.g. realistic job preview) Do selection criteria reflect the KSAs and cultural values of the service and unit where the role is located when technology is implemented? Are changes required to basic and advanced training? (e.g. to incorporate the development of new skills and processes ) Do management tools need changing or developing? (e.g. to reflect the service and unit culture where the technology is implemented) Does the pay and rewards structure accurately reflect the nature of the role of units, groups and individuals? (e.g. do roles become more technical)
342
P. Sirett et al.
Summary There is always a temptation to apply new technology to current organisations and practice rather than to identify how new or emerging technology can offer opportunities for fundamental change. There is much historical evidence that points to the advantages of embracing change and to the penalties of not doing so. This study focused on the potential of the network in the knowledge that, within the commercial sector, the network has revolutionised business practice, culture and structures, and that it is the main overarching change within the UK force structure. The study acknowledged revolutionary rather than evolutionary analysis of the S&O impacts. The notion of ‘disruptive’ rather than ‘sustaining’ innovation reflects this approach, and the study attempted to discriminate between those new technologies that may offer a step-change in terms of capability and those that do not. The S&O criteria tables represent an important output as they offer a guide to how further detailed studies might be conducted in relation to particular technologies or disciplines. The S&O criteria should be used as a starting point to determine the potential impact of integrating emerging technology, especially that considered revolutionary or disruptive. The S&O domain needs to be examined from a corporate perspective adopting a holistic approach to the network and not be confined to an individual equipment programme as this will support harmonisation across the organisation. The profile of HFI, within the capability development process, may not be sufficiently prominent to address the more revolutionary issues that may arise in the S&O domain. The HFI process is very much geared towards an equipment focus and there may be merit in re-examining the place of HFI within the concepts-to-capability approach. The combination of the changing nature of future conflict and the revolutionary opportunities offered by NEC indicates that a study of the S&O impacts of such emerging technology is most timely and appropriate. The S&O impacts of a network in the commercial field have been considerable and may merit a separate study in order to extrapolate those high-level S&O issues that relate to successful organisational change.
References Christensen, C.M. (1997) The Innovators Dilemma: When new technologies cause great firms to fail. Harvard Business School. Boston Crebolder, J.M., Cooper-Chapman, C.S., Hazen, M.G., Corbridge, C. (2007) Assessing Human Performance in a Distributed Virtual Battle Experiment. 12th International Command and Control Research and Technology Symposium. Malvern, England Gibson, L.G., Ivancevich, J.M., Donnelly, J.M. (1994) Organisations; Behaviour, Structure, Processes (4th Ed.) Irwin. Boston, Massachusettes, USA Kirke, C (2004) Organisational Culture: The Unexpected Force. Journal of Battlefield Technology Vol 7, No 2, pp 11–17
Criteria for assessing social and organisational impacts
343
Leedom, K.D., Egglestone, R.G., and Ntuen, C.A. (undated) Engineering Complex Human–Technological Work Systems – A Sensemaking Approach. Evidenced Based Research Inc. Vienna Mathieson, G., and Dodd, L. (2004) A conceptual model of organisational and social factors in Headquarters. Command and Control Research and Technology Symposium: The Power of Information Age Concepts and Technologies. Malvern, England McKenna, E. (2006) Business Psychology and Organisational Behaviour (4th Ed.). Psychology Press. Essex Oden, K.B., Terrence, P., and Stahl, J. (2006) Social Influence in a Distributed Simulation. Proceedings of the Human Factors And Ergonomics Society 50th Annual Meeting. San Francisco, USA Pierce (Capt), T. (2004) Warfighting and Disruptive Technologies — Disguising Innovation. Frank Cass Publishing. Oxon. England Schein, E.H. (1990) Organisational Culture. American Psychologist, 45, pp 1091–19 Schein, E.H. (1996) Three cultures of management: The key to organizational learning. Sloan Management Review, 38(1), pp 9 Smither, R.D. (1994) The Psychology of Work and Human Performance (2nd Ed.) Harper Collins Publishers Inc. New York The MOD HFI Process Handbook Edition 1 December 2005 p 2 Turner, P. (2007) The Role of Sensemaking in the Command-ISTAR relationship. 12th International Command and Control Research and Technology Symposium. Malvern, England Warne, L., Ali, I., Bopping, D., Hart, D., and Pascoe, C. (2004) The Netric Centric Warrior: The Human Centric Dimension of Network Centric Warfare. Defence Systems Analysis Division. Defence Science and Technology Organisation. Australia. p 41
INSPECTION
ANIMATING THE INANIMATE: VISUAL INSPECTION FROM THE MEDICAL TO THE SECURITY DOMAINS A. Gale Applied Vision Research Centre, Loughborough University, Loughborough, LE11 3UZ, UK Inspection performance in medical imaging has long been studied with an original concentration upon the analogue chest X-ray image. However, in recent years much of this research impetus has shifted to the breast screening domain where this rare disease poses intriguing inspection issues and where digital imaging has introduced different inspection scenarios as well as the use of computer decision aids. Additionally there has been an expansion with many different imaging modalities. Airport passenger baggage security screening has many similarities to medical imaging in that it uses X-rays to produce a digital image which is then inspected for rare targets. Extrapolating research findings from medical imaging to airport screening is potentially useful then in helping to understand this domain.
Introduction The process of examining human performance in the interpretation of medical images has a very long history and researchers some 60 years ago found that there was a significant amount of inter-observer as well as intra-observer disagreement when examining X-ray images (Yerushalmy, Harkness, Cape & Kennedy, 1950; Garland, 1949). It would be envisaged that one would find a decreasing trend of errors reported in the literature emanating from the following half century of different types of experimental investigation into radiologists’ performance in examining a large range of different types of medical image and the numerous publications. Unfortunately, however, these all find that no matter how expert the observer – s/he still makes errors (Scott, Gale & Hill, 2008). This unfortunately gives rise to litigious actions from patients which provides a steady source of income to some lawyers (Berlin, 1996). Making errors, by implication, means that errorless performance can be achieved and much of the published research has gone into trying to understand why errors occur and then devise ways in which these could at least be minimised. Until fairly recently all medical images were composed of X-ray films which were examined on some form of illuminator in a darkened room. Chest X-rays used to be the most common image class studied. Moveable blinds on these viewing boxes and multiviewers (which allowed for many images to be examined simultaneously) permitted the removal of extraneous light emanating from the viewing box which otherwise caused veiling glare. Many investigations studied how chest X-ray images were 347
348
A. Gale
examined, chiefly because of their inherent size these were difficult to visually inspect. Eye movement studies demonstrated repeatedly that such images were examined very quickly (within 10–20 seconds) and that the observer concentrated only upon certain areas of the image. Radiological experience and expertise caused such eye fixations to be located at high probability areas of the image for finding an abnormality. Instructions or information given to the observer prior to examining the image (e.g. ‘this person has had a road traffic accident’) resulted in the observer having an increased likelihood of detecting some abnormality related to the instructions (e.g. a broken rib) but decreased the likelihood of detecting other abnormalities which were visibly present. An oft reported finding which was particularly troubling was that fairly obvious (conspicuous) abnormalities could be easily missed. Satisfaction of search, where once some abnormal features has been detected then the image inspection is terminated so leading to other abnormal features being missed. A theoretical model, based on experimental and cognitive psychology research findings, was developed (Kundel, Nodine & Carmody, 1978; Gale, 1993) that argued that medical image inspection comprised three key processes: visual search, detection and interpretation. Simply put – saccadic eye movements serve to shift visual attention over the image causing some image areas to be seen in foveal vision where these areas are processed in detail. Ideally this process gives rise to such image features being detected which areNow that the cognitively amassed together and recognised as some image related attribute which may indicate an abnormality. Errors can occur in one or more of these processes at the same time but importantly it is possible to tease out for an individual in a particular situation just what is causing their errors (e.g. faulting scanning, poor knowledge of abnormality appearance). This then means that training can be developed which instead of simply being generic in nature and being offered to all observers can be very targeted to the specific needs of each individual. The model has stood the test of time chiefly because it is both detailed enough to be practically useful and general enough to be applicable.
Modern radiology Today radiology is replete with a wide range of imaging modalities and the X-ray film image is all but consigned to obscurity. The modern radiology department is filmless and comprises multiple imaging systems which produce digital images and transmit these, together with patient information and radiologists’ reports both across hospitals, as well as across countries. Hospital IT systems include PACS (Picture Archiving And Communication Systems), which transmit images, and RIS (Radiological Information Systems), which transmit information, working together to achieve this. Established international standards such as DICOM (2003) for images and image display systems allow for interoperability between different manufacturers’ equipment in an almost ‘plug and play’ scenario. Images can be viewed on very high resolution monitors in controlled viewing environments for diagnostic imaging, on projection display systems for multidisciplinary meetings,
Animating the inanimate
349
and on lower resolution monitors for other information purposes for instance in operating rooms or on hand-held devices such as by the patient’s bed. Medical imaging now comprises a very wide range of dynamic human imaging modalities which besides the standard X-ray imaging includes: computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET – a nuclear medicine imaging which produces a three-dimensional image of functional processes), MRI/PET(anatomical changes are shown by MRI and functional information illustrated by PET), Single photon emission tomography (SPECT), functional magnetic resonance imaging (fMRI), SPECT/CT, CT Colonography, Ultrasound, & Tomosynthesis. These techniques produce digital images in two, three or four dimensions which can be both static or dynamic in nature. Additionally, volumetric rendering of images is often undertaken which produces computer-enhanced 3D/4D images in which predefined structures can be removed, or added, as a whole which enhances the identification of abnormalities and images can also be rotated and manipulated in three/four dimensions which aids both diagnosis and subsequent surgical planning. Despite such complexity in images inspecting these images and reaching a diagnosis is still achieved quickly; 3 and 4D images are inherently slower to report as one has to scroll through many image planes in a stack of individual images to identify an abnormal structure.
Breast screening Breast screening is a particularly interesting imaging domain which has been the last to become digital and in the UK the national introduction of digital screening is currently undergoing trials with it being implemented in 2010. Some twenty years ago the UK decided to introduce breast cancer screening for all women aged 50–64 so that each woman would be screened every three years. This vitally important national programme was closely coupled to discoveries in other countries of the advantages of screening women in this age range. Screening involves taking two X-ray views of each breast and then examining all four X-ray images together on a multi-viewer. Where a woman is being screened on a subsequent screening round then her current four images are examined and compared for any changes to her previous four images taken three year previously. Screening is not medical diagnosis and so in screening a simple binary decision has to be taken by the radiologist as to whether to recall the woman for further investigations or if she appears normal then to simply re-screen her in three years time. The difficulty for the radiologist in examining screening images is that the incidence of breast cancer is very low. The latest published data (Patnick, 2007), for 2005–2006, shows that out of almost 1,900,000 women screened only some 4.7% were recalled for assessment with some 14,841 cancers detected. Thus the radiologist has the difficult task of trying to identify very rare signs of potential cancer at their earliest presentation (which can mean almost pin-head sized image changes). Cancer can occur anywhere within the breast image which can be masked by the image background density which varies with the woman’s age.
350
A. Gale
To improve the detection of early disease double reading is advocated where two individuals examine the images at different times and (ideally) do so unaware of the other’s decisions. Any disagreements are then arbitrated by a third person of a consensus opinion is reached. To aid the radiologist in this task much research has been devoted to the investigation of CAD (Computer Aided Detection) systems which pre-process the image and then when the radiologist views it s/he can call up an image overlay which displays various cues on the image that the CAD system considers to be suspect and indicative of abnormality (e.g. Karssemeijer, Otten, Rijken & Holland, 2006). Only a few years ago such systems were producing many false positive responses but nowadays the accuracy of these systems has been carefully improved and various systems are now FDA approved for routine clinical usage. Interestingly, a recent UK study has shown that a radiologist working with a CAD system can perform just as well as two radiologists separately reading the same case. In screening, as a diagnosis is not being arrived at but simply a decision of whether to recall a woman or not, then other professionals have been introduced into the image interpretation arena. Primarily these are highly trained radiographers, who traditionally have been involved with taking the original image and ensuring the high technical appearance of that image. Research, as well as everyday experience, indicates that such individuals can detect cancer just as well as experienced radiologists (Scott & Gale, 2007).
Airport screening The security screening of passenger bags is a key measure to detect both contraband and terrorist threats to safe air travel. Both passenger carry-on bags and bags carried in the airplane’s hold are passed through a tunnel where these are X-rayed and the resultant digital images examined by an operator using one or two monitors (Gale, 2005). Security inspection at airports is in many ways similar to medical image inspection – both techniques produce images which may contain a rare target and must be reported in a timely and very accurate manner. The airport images are particularly similar to breast screening images as in both situations one is dealing with very rare target occurrence. Airport imaging of baggage items typically produces 2D digital images, with some 3D imaging now being introduced. This development is important as a single X-ray viewpoint results in an image with overlaying features which can both mask a potential target or else the overlaying features can imply a target presence. Whilst images of the hold bags are examined in a secure and controlled environment the carry on bag images are examined by a security officer sitting beside the X-ray scanner. If a suspicious item is found then the bag can be further examined by hand or if an explosive substance is suspected then the bag can be left within the X-ray tunnel and help summoned. This screening situation allows easy communication between the screener and other security personnel who may further search the bag, although it is probably not the best environment for detecting rare targets in the bag images due to noise, lighting and other potential distracting environmental
Animating the inanimate
351
factors which can affect the screeners’ performance. Medical imaging research has demonstrated that difficult to perceive radiographic features can be aided in their perceptability by examining these images in a darkened room with a low level of ambient lighting. However, whilst in breast screening research has demonstrated which radiographic features are key early signs of abnormality, in airport screening the target, predominantly a knife, gun or IED (improvised explosive device), can be of any shape and size and so it is much more difficult to identify key threat feature elements. Furthermore, terrorist threats inherently change and develop thus, for instance, until mid 2006 there was little thought about the carrying on to aeroplanes of liquids, however since that time the volume of liquids able to be transported has been severely curtailed. There is not the same volume of research into airport screening as there is with medical imaging. However, many of the findings from medical imaging can translate to the other domain. For instance, the same theoretical model has been applied and found useful in examining target detection (Liu & Gale, 2007a) and in developing training (Liu & Gale, 2007b). It has also been applied to the study of expertise development from naïve observers to more experienced screeners (Liu, Gale & Purdy, 2006). Gale, Mugglestone, Purdy, and McClumpha (2000) demonstrated that most airport screening errors in an experimental situation were due to interpretation rather than detection or visual search errors. Furthermore, the proportion of such errors were more prevalent than are typically found from medical imaging research. In airport screening there is pressure on the screener to inspect images quickly due to the need to maintain timely passenger throughput and the effects of time pressure on searching such images has also been investigated (Liu & Gale, 2007). In medical imaging whilst there is always pressure to inspect images quickly, this is not quite the same as in airport screening where an immediate decision has to be reached. Bag images are not stored for any future use as they are in medical imaging. As with medical images, computer aids can readily help the airport screener and these can be applied to enhance potential organic (possible explosives) material and other image characteristics. The screener has available a range of such aids. However, the image processing capabilities have not quite reached the precise nature of those that are found in medical imaging. For instance, image processing algorithms could be used to delineate specific items within each bag and determine whether these reflect known normal similar item feature characteristics. Whilst ongoing research is investigating such matters the software is not widely implemented commercially. In medical imaging the clinicians have to undergo considerable detailed training which covers various relevant domains (e.g. human physiology) over several years coupled with actual image inspection training. Although other professions (e.g. advanced practitioner radiographers, physicians, surgeons) as well as radiologists are now involved in reporting images they all have considerable medical training. In contrast, in airport screening almost any security officer could potentially become a screener. Consequently, research has investigated what skills are required in the
352
A. Gale
airport scenario (Hofer & Schwaninger, 2004) and a test of screening ability has been developed which can be used to aid airports in the selection of screeners (Hardmeier, Hofer, & Schwaninger, 2005). Breast screening research has demonstrated that the dual reading of images significantly reduced the likelihood of making errors; such dual reading is not employed in airport screening. However, in order to maintain screening performance levels, false TIP (Threat Image Projection) targets can be added to the everyday screening bag images and then feedback is provided to the screener when they detect such targets (Hofer & Schwaninger, 2005). Research has also demonstrated that pre-employment tests can help select appropriate individuals to be security screeners.
Conclusions Medical imaging has a longstanding and massive research history in understanding how clinicians inspect these images and new technology is aiding them in reaching appropriate diagnostic decisions. Still errors can occur in this complex domain and it is most unlikely that errorless performance can be achieved. However, it is important that each individual inspects images to the best of their abilities, that their abilities are known, monitored and that ongoing training is undertaken. There are many similarities with the airport baggage screening task where again errorless performance is probably not achievable. It is argued that airport screening can be improved by determining whether factors found important and useful in reducing errors in medical imaging can translate appropriately into airport screening. Computer aided detection of abnormalities is well advanced in the medical domain and there seems to be room for security to progress to the same degree. Human frailty in such inspection tasks needs to be acknowledged and aids developed which truly assist each individual inspector appropriately.
References Berlin L. Malpractice issues in radiology. 1996, American Journal of Radiology, 167, 587–590. DICOM ACR-NEMA. 2003, Standards Publication No. 300–1985. Gale A.G. 1993, Human response to visual stimuli. In Hendee W. & Wells P. (Eds.) Perception of Visual Information, (New York) Springer Verlag. Gale A.G. 2005, Human factors issues in airport baggage screening. In P.D. Bust & P.T. McCabe (Ed.) Contemporary Ergonomics 2005. (Taylor & Francis, London), 101–104. Gale A.G. Mugglestone M., Purdy K.J., McClumpha A. 2000, Is airport baggage inspection just another medical image? In Krupinski E.A. (Ed.) Medical Imaging: Image Perception and Performance. Progress in Biomedical Optics and Imaging, 1, 26, 184–192. Garland L.H., On the scientific evaluation of diagnostic procedures. 1949, Radiology, 52(3), 309–28.
Animating the inanimate
353
Hardmeier, D., Hofer, F., & Schwaninger, A. 2005, The x-ray object recognition test (x-ray ort) – a reliable and valid instrument for measuring visual abilities needed in x-ray screening. IEEE ICCST Proceedings, IEEE, 39, 189–192. Hofer, F. & Schwaninger, A. 2004, Reliable and valid measures of threat detection performance in X-ray screening. IEEE ICCST Proceedings, IEEE, 38, 303–308. Hofer F., & Schwaninger, A. 2005, Using threat image projection data for assessing individual screener performance. WIT Transactions on the Built Environment, 82, 417–426. Karssemeijer, N, Otten, J.D.M., Rijken H, Holland R. 2006, Computer aided detection of masses in mammograms as decision support. British Journal of Radiology, 79, S123–S126. Kundel, H.L., Nodine, C.F., Carmody, D. 1978, Visual scanning, pattern recognition, and decision making in pulmonary nodule detection. Investigative Radiology, 13:175–181. Liu X., Gale A.G. and Purdy K. Development of Expertise in Detecting Terrorist Threats at Airports. 2006, In D. de Waard, K.A. Brookhuis, and A. Toffetti (Eds.) Developments in Human Factors in Transportation, Design, and Evaluation, Maastricht, the Netherlands: Shaker Publishing. 329–332. Liu X., Gale A.G., Purdy K. and Song, T. 2006, Is that a gun? The influence of features of bags and threat items on detection performance. In Bust P.D. (ed.) Contemporary Ergonomis 2006, (London, Taylor and Francis), 17–22. Liu X. and Gale A.G. 2007, Effects of time pressure on searching for terrorist threats in X-ray air passenger luggage images. In D. de Waard, R. Hockey, P. Nickel, and K. Brookhuis (Eds.) Human Factors Issues in Complex System Performance. (Maastricht, the Netherlands: Shaker Publishing). 435–442. Liu X. and Gale A.G. 2007, How visual skills and recognition ability develop with practice in Airport Luggage Inspection. In Bust P.D. (ed.) Contemporary Ergonomics 2007, (London, Taylor and Francis). 196–202. Patnick, J. NHS Breast Screening Programme Review 2007. (NHSBSP, Sheffield). Scott H.J. & Gale A.G. 2007, How much is enough: factors affecting the optimal interpretation of breast screening mammograms. In Y Jiang and B Sahiner (Eds.), Image Perception, Observer Performance, and Technology Assessment. Proceedings of SPIE 2007. Scott H.J., Gale A.G. & Hill S. 2008, How are false negative cases perceived by mammographers? Which abnormalities are misinterpreted and which go undetected? In D. Manning and B Sahiner (Eds.), Image Perception, Observer Performance, and Technology Assessment. Proceedings of SPIE 2008. Yerushalmy J, Harkness, J.T., Cape, J.H. & Kennedy B.R. 1950, Role of dual readings in mass radiology. American Revue of Tuberculosis, 61, 443–464.
INSPECTING FOR SAFETY HAZARDS K. Woodcock School of Occupational and Public Health, Ryerson University, Toronto, Canada Inspection is a common element of safety management systems, controlling hazards by early detection and correction. However, little relevant literature is available. In this study, mixed methods were used including participant-observation of amusement ride safety inspection to describe and model the context and approach to inspection, and a survey of 105 safety inspectors across three sectors to examine the fit of the description. Safety inspection was seen to be multisensory, presenting potential defects in varied configurations, and making decisions more nuanced than accept/reject. Distributed cognition including peer consultation is used to deal with uncertainty, but feedback is weak or absent. Implications for performance supports are discussed.
Introduction Inspection is a common element of safety management systems, controlling hazards by early detection and correction. As has been widely recognized, accidental injury data can only be collected after injuries occur, and valid analysis is difficult because they are statistically rare events (Woodcock 1989). In contrast, safety inspections identify required corrections as well as appraising the extent to which safety is compromised. Inspection is a part of internal safety management systems as well as external or enforcement systems. Although seemingly a ubiquitous practice, the safety science literature does not discuss human factors affecting inspection performance. A literature search of an electronic database comprising human factors, ergonomics, psychology, and safety science literature 2000–2007 for “safety + inspection” found few papers relevant to safety inspection, none addressing inspector performance. Most of the literature about safety inspection is normative, for instance, that safety inspection performed a certain way predicts program-level safety outcomes such as injuries (Laitinen et al., 1999). Occupational safety textbooks describe inspection as though the actual search and assessment of defects is self-evident. The emphasis is on the need to plan the inspection, incorporating information about prior accidents, a review of operations and potential accidents, the hazard-identification input of workers, and applicable standards (Reich 1986, Mansdorf 1993). While texts commonly describe principles and standards related to a variety of hazard types, the implication is that the inspector 354
Inspecting for safety hazards
355
will intuitively know how to identify and correctly assess those hazards when encountered. Expectations for safety inspection are high. Public discourse about safety inspection reflects the expectation that inspectors will detect anything unsafe and the perception that the task is simple. For instance, at the time of writing, a propane handing facility had recently exploded in Toronto Canada, temporarily displacing over 10 000 local residents from their homes and dispersing asbestos contamination. Despite the size of the industry and the small rate of safety failures, the media and public rush to judgement disparaged the overseeing agency. Politicians supported this position by pointing out “hazards” at approved sites. Although the purported “hazards” were mostly cosmetic, the exercise suggests a perception that safety inspection is an intuitively obvious process requiring little more than diligence. This paper focuses on third-party inspection as performed in many settings including occupational health and safety, public health, and technical standards inspection. While inspectors often examine or recommend management systems such as training, policies and procedures, the purpose of those management systems elements is to achieve some physical safety through physical conditions or execution of particular behaviours. This paper therefore focuses on inspection of conditions and operation, as this verifies the fulfilment of the management-system intentions as well as the achievement of safety objectives.
Safety inspection in the context of “inspection” in human factors The task referred to as industrial inspection in the HF literature focusses on quality, such as in manufacturing and service delivery. Inspection is modelled as a series of stages from preparation to follow-up. The core of the inspection process is the search for potential defects and decision whether to accept or reject specific observations (Drury & Prabhu, 1996). Safety inspection similarly involves a search for potential defects and decisions about acceptability. Is “safety inspection” the same task as inspection, or is the use of the same word merely a homonym for a distinct task? If so, what use can be made of the Human Factors legacy research on inspection? Are any of the recommended performance supports used or useful? Are the research methods relevant? If “safety inspection” is different, how widely do its characteristics apply? In the (quality) inspection task, defect types are often completely and objectively defined and described prior to assignment of the task to the inspector, based on the operational requirements such as quality of manufactured product or completion criteria for a maintenance job. The inspector’s task is search, recognition, and decision. As a result, signal detection theory can readily be used to study inspection performance and evaluate performance supports. Decisions can be classified as hits (correct rejection of defect), misses (acceptance of defect), false alarms (rejection of acceptable condition) and correct acceptance of non-defective condition by comparing actual decisions to correct decisions. Drury and Prabhu (1996) described aircraft inspection as a less predefined task incorporating inspectors’ own experience, training, distributed knowledge, and mental models in shaping the
356
K. Woodcock
search agenda, but retained a closed-loop model. Misses and false alarms can be due to ineffective visual search, defect perception, and misjudgement. In safety inspection, the inspector seeks to recognize hazards in a device, place, or process embedded in its natural environment. Generally, all defects must be found and corrected to restore the safety of the operation. The safety inspector must often physically reposition himself to collect the necessary observations, requiring him to know where to go as well as how to make the observations. Other than as a broad spectrum of generic defect types, hazards are often ill-defined. For instance, cracks or defective welds can be defined as types of defect, but may be found in different places on different devices, and may be critical on some and cosmetic on others. Inspectors are also looking for potential mechanisms whereby certain broadly defined outcomes can occur, for instance falls, lacerations, or device collapse. Rather than defining a defect prior to inspection and then tasking an inspector to search for it, the safety inspector is tasked to identify potential instances of hazards that are often not fully defined, but which could increase the chance of some adverse outcome. Without a finite set of objectively defined defects, signal detection theory cannot be applied. The closest alternative to a “correct” judgement is comparison with the judgement of an accepted expert. Ultimately, however, the expert may have no absolute validation of her own judgement because the decision is often not a closed loop. While investigation findings or trouble reports may communicate inspection points and the way defects might be manifested and detected, there may be minimal knowledge of results for individual inspections. Identified “defects” will be corrected even if objectively it would be considered a false alarm, and unless an inspection was skipped, failures are typically linked to subsequent deterioration rather than inspection misses. Inspected “items” – e.g., devices, places, and the activities of people – are diverse and often one-of-a-kind. This variability requires what Rasmussen (1986) described as a topographic diagnostic search in which the operator compares the whole of what is currently being observed to a recollection or impression of a normal version. However safety inspectors may have no recollection of a normal version. For example, an amusement ride inspection commonly would cover many different rides on the same midway. Assignment based on familiarity with a specific device is infeasible. Inspectors must construct a “normal” from abstract knowledge, subjectively similar devices perceived to be valid comparators, and other sources. The inspector must be a generalist to recognize and evaluate any of a broad spectrum of hazards in an unfamiliar device, place, or operation. The present policy environment favours risk-informed decision-making (Etherton 2007, TSSA 2007). In this practice, safety inspectors may give owners time to correct a nominal defect with minimal safety implications, while more serious defects would be addressed more strictly. Discretion imposes further complexity, particularly for novices. The novice must not only recognize indicators of defects but often must also be able to legitimize a decision based on risk. Quality inspection may also include a variety of skill-based, rule-based and knowledge-based decisions (Drury & Prabhu 1996). However, the inspector’s search for defects may be simplified by predefining a finite list of defects. Search is also simplified when the inspected field has some uniformity or standard
Inspecting for safety hazards
357
presentation; perhaps items have a consistent configuration, and come to the inspector at an inspection station. Performance can be enhanced by learned strategies. In some inspections, all concurrent defects should be found, as the unit will be reworked. In others, the first defect results in scrapping the unit. In the latter case, inspections can be optimized by searching for defects in order of probability. Other senses are mentioned but the HF literature has focused on visual search, visual flaw detection and decisions (Drury 2001, Drury & Prabhu 1996, Gramopadhye, Drury and Prabhu 1997, Gramopadhye, Drury & Sharit 1997). Inspection research has explored methods to support performance (Chabukswar, et al., 2003, Gramopadhye, Drury & Sharit 1997, Mehta, et al., 2005). Unfortunately, with numerous, ill-defined and variably located defect types, most of these performance enhancements do not seem practical for safety inspection. The goal of this paper is to explore and begin to model safety inspection and identify characteristics and strategies that could be leveraged to support safety inspection performance.
Method This study used mixed methods. The first portion was an immersive participantobservation study of amusement ride safety inspection, including over 50 days of observations over four and a half years, in which the researcher became a trainee with expert mentors and novice inspector peers and participated in group hands-on inspection training and actual inspections including group training (three sessions with 50+ trainees and three sessions with 200+ trainees) and observed actual inspections with five individual inspectors. Inspectors were familiar with this mentored structure, thus introducing the researcher as a novice trainee allowed naturalistic observation. Development of the model was iterative and holistic, with emerging impressions tested in the form of hypothetical questions from a novice trainee. The fifth-year observations were used to challenge and make final refinements to the proposed model and summarize generally noted strategies. Secondly, a self-report survey was used to examine the model and strategies quantitatively using feedback from professional amusement ride inspectors. The survey was also administered to workplace safety inspectors and public health (food safety) inspectors, to explore the potential generalizability to safety inspection in other domains. The survey was administered anonymously online to voluntary respondents solicited through work groups (workplace and ride safety) and a professional organization (public health). Questions used ordinal scales. Non-parametric tests of significance (Kruskal-Wallis) were used to compare among sectors and scenarios.
Results Observations and model derived from amusement ride inspections Rule-based decision-making can readily be used to reject an intolerable discrepancy or violation of explicit regulation. However there is first a challenge of locating
358
K. Woodcock
Figure 1.
Model of approach to safety inspection.
defects in a complex field, and in many cases a more complicated knowledge-based decision required to determine whether to accept or reject the condition. Rather than searching for specific defects, ride safety inspectors performed a topographic search using holistic observations, forming impressions of the defect types that might be present, searching both for observable defects and broad visual, auditory and kinaesthetic patterns that the inspector associates with safety deficiencies – perhaps fresh paint, excessive grease, holes, puddles – or certain structural or mechanical features that the inspector recognizes as prone to wear or failure or as being linked to a service bulletin or accident. The model (Figure 1) differentiates the identification of a potential concern from the acceptance/rejection decision to identify it as a “defect”. Uncertainty and its resolution was a common part of the inspection decision process. With gradual wear and cosmetic deterioration, the perceptible difference between defective and acceptable may be slight. Where uncertainty was simply perceptual, potential defects were often probed tactually, visually and aurally after improving the inspector’s view or contact with the target. Uncertainty also arose when the safety inspector did not have recollection or knowledge of the normal system and constructed expectations from general knowledge and similar systems. To resolve uncertainty as to whether the observation was a defect warranting correction, a common strategy was to search for permissive sources such as manufacturers’ bulletins, and if finding none, to identify a condition as a discrepancy. Inspectors also constructed comparators by examining similar components within the device. While consistency did not establish acceptability, inconsistency indicated a discrepancy and potential hazard. Ride safety inspections often entailed multiple concurrent tasks, interleaved due to interruptions, delays, and spontaneous observations, providing additional potential comparisons between subjects of concurrent tasks. Inspectors used distributed cognition, sharing knowledge with colleagues, to learn methods to obtain evidence to resolve uncertainty.
Inspecting for safety hazards
359
Along with basic technical knowledge pertaining to dozens of generic defects, training often included mentoring of novices by experienced inspectors to transfer tacit knowledge needed to generate mental models of stress points and failure modes. Knowledge transfer included substantial hands-on, literally tactile exposure to various devices, and commonly took an episodic form, as anecdotes established proof-of-concept of specific failure modes. Experienced inspectors gave higher ratings to the usefulness of manuals, investigation reports, reminder notes, experience and even anecdotes from others than novices did (Woodcock 2004). Exposure to failure case studies (first-hand or second-hand) provides not only knowledge but also a framework to assimilate information from these resources. To counter resistance to enforcement, experienced inspectors were able to cite direct and indirect knowledge of past failures as proof-of-concept for failure modes. Novices may lack the direct knowledge but formulated indirect knowledge from resources such as prior inspection reports indicating that the particular condition had previously been determined to be a legitimate defect to correct. Although checklists are a common support for many types of inspection (Ockerman & Pritchett, 2000), ride safety inspectors were not seen to use checklists concurrently due to the physical demands of inspection requiring them to climb on, under and through the ride. Also, most checklists encountered were organized by type of component, in contrast to the usual area-by-area approach to inspection dictated by physical efficiency, as also reported about aircraft inspections (Larock & Drury 2003). The common topographic search effectively uses the device itself as a “checklist”, progressing from part to adjacent part. Inspectors used generic knowledge and mental models of ride action to determine stress points and defect types. Paper checklists were used, but their use was consecutive: before inspection, along with reference materials and device demonstrations, to develop mental models when a device was unfamiliar, and after inspection to confirm that issues earmarked for inspection had not been missed.
Survey A survey response was received from 105 inspectors (53 food safety, 38 workplace, and 14 amusement ride), with roughly 84% of each group submitting substantially complete responses. Completed questions from partial responses were included in the analysis. Due to the indirect method of invitation, it is not possible to know the total number invited to respond or population size. The survey responses (means) are tabulated in Table 1. (The table paraphrases the statements; a verbatim version is available on request from the author.) The combined mean is shown where there is no significant difference between sectors. The sector mean is shown separately where the sector significantly differs from the other sectors, and the combined mean in that case excludes the differing sector. Except for ordering independent analysis (mean 2.98 on a scale from 1 = Never to 5 = Always), the response to identified defects varied significantly by the nature of the associated rule (p < .001). Consultation was rated higher when the defect was uncertain, while compliance immediately or on risk-based timelines was more common for quantitative and objective qualitative rules. Owners sometimes protested
360
K. Woodcock
Table 1.
Inspectors’ responses to survey.
Statement/description
Mean1
10–20 defect types 0 or so defect types 00 or more defect types Defect types were known prior to inspections Defect types added during inspection On arrival onsite, “take a moment” for general overview . . . . . . .only when process is unfamiliar Any basically qualified inspector could be assigned
3.68 2.55 1.81 3.73 3.43 4.43 3.08 3.97
Expertise is considered in choosing who is assigned to inspection
More-expert inspector accompanies when lacking specific expertise
When accompanying more experienced, expected to learn from
3.73
Importance of . . . vision (1 = not important to 3 = primary) . . . of verbal hearing . . . of smell, non-verbal hearing . . . of touch
2.93 2.15 1.98 1.94
Have checklist
If have one, use checklist point by point . . ., use checklist to prepare . . ., use checklist afterward to ensure no omissions
2.48 2.83 3.64
Encounter defects covered by quantitative rules
. . . defects covered by qualitative but objective rules . . . defects unspecified but covered by general rules . . . potential defect with uncertainty whether it is a hazard
3.64 3.32 2.8
Allow time to comply based on estimated risk Consult colleagues on whether to order Order immediate compliance Consult colleagues on time to comply Order owner to obtain independent analysis
3.81 3.4 3.06 2.78 3.23
Comfortable with the amount of discretion required Less-experienced inspectors are (1 stricter, 5 more lenient) More-experienced inspectors are (1 stricter, 5 more lenient)
4.36 2.56 3.49
Inspection misses are identified in failure investigations Inspection misses are caught by colleagues during inspection Deterioration is so gradual that next inspection could catch miss Deterioration is so rapid that failure investigation could not determine whether there was an inspection miss
2.63 2.42 2.93 3.08
Sector(s) differing2
p
Work 3.17 Work 3.97
.006 .005
Food 1.68 Work 2.92 Rides 3.58 Food 1.98 Work 2.56 Rides 2.92
<.001
<.001
Work 1.62
.001
Food 4.27 Work 1.78 Rides 3.86 Rides 3.69
<.001
<.001
Food 4.43 Work 3.34 Rides 3.83 Food 4.20 Rides 3.92
<.001
Work 3.45
.001
Food 2.69
<.001
Rides 3.00
.031
.005 .023
Notes: 1. Range: 1 = Never to 5 = Always, unless indicated. Mean rating pools all groups not identified as significantly different (p > .05). Significance test uses non-parametric Kruskal-Wallis tests (2 degrees of freedom among 3 groups; 1 d.f. between 2 groups). 2. “Food” refers to public health food safety inspectors, “work” to workplace health and safety inspectors, and “rides” to amusement ride inspectors.
Inspecting for safety hazards
361
orders (2.35); both threats (2.02) and offered benefits (1.44) were rare. Inspectors were less likely to consider a protest based on claim of hardship or unfairness (2.22) than when the owner provided contrary evidence (3.08). This varied by sector (p < .01) and by type of rule (p < .01). Food premise inspectors gave more receptive ratings to hardship protests, while ride inspectors reported considering contrary authorities for quantitative and qualitative rules. Uncertainty was generally associated with more consideration for owners’ protests.
Discussion Participant-observation suggested and the survey confirmed that the defect types considered in safety inspections are more numerous and less predefined than quality inspection studied in the HF literature. Safety inspection is multisensory and requires decisions more nuanced than accept/reject. Distributed cognition including peer consultation is used to deal with uncertainty, but performance feedback is weak or absent. The general similarity of survey responses among the three sectors suggests that safety inspection is a distinct class. Using mixed methods complemented the researcher’s participant-observation interpretation with survey data from practitioners. The perception that lessexperienced inspectors are stricter and more-experienced inspectors are more lenient reinforces the importance of expertise in the risk-based decision environment. Further study is warranted to validate the model of the inspection approach and examine how to efficiently resolve uncertainty and make optimal risk-based decisions across all levels of expertise. The signal-detection theory model used for quality inspection studies is infeasible with the ill-defined open-loop task. A viable feedback mechanism through investigation would be an option to close the loop. While the diversity of inspection environments limits the opportunity to thoroughly predefine the task for training and checklists, safety inspectors would likely benefit from supports to reduce uncertainty, such as reference material linking defect indicators and failure.
Acknowledgements A Discovery Grant from the Natural Sciences and Engineering Research Council (Canada) supported this work. The Technical Standards and Safety Authority of Ontario provided valuable assistance with field observations. The survey distribution assistance of three inspection groups is appreciated.
References Chabukswar, S., Gramopadhye, A.K., Melloy, B.J. & Grimes, L.W. 2003, Use of aiding and feedback in improving visual search performance for an inspection task. Human Factors and Ergonomics in Manufacturing 13, 115–136.
362
K. Woodcock
Drury, C.G., & Prabhu, P.V. 1996, Information requirements of aircraft inspection: framework and analysis. Int. J. Human-Computer Studies, 45, 679–695. Drury, C.G. 2001, Human factors and automation in test and inspection (Chap. 71). In G. Salvendy (Ed.) Handbook of Industrial Engineering. 3rd ed., (Wiley & Sons). Etherton, J. R. 2007, Industrial machine systems risk assessment: A critical review of concepts and methods. Risk Analysis, 27, 71–82. Gramopadhye, A.K., Drury, C.G. & Prabhu, P.V. 1997, Training strategies for visual inspection. Human Factors and Ergonomics in Manufacturing 7, 171–196. Gramopadhye, A.K., Drury, C.G. & Sharit, J. 1997, Feedback strategies for visual search in airframe structural inspection. Int. J. Industrial Ergonomics 19, 333–344. Laitinen, H., Marjamäki, M., & Päivärinta, K. 1999, The validity of TR safety observation method on building construction. Accident Analysis and Prevention 31, 463–471. Larock, B., & Drury, C.G. 2003, Repetitive inspection with checklists: Design and performance. In P. McCabe (Ed.), Contemporary Ergonomics 2003, (Taylor & Francis). Mansdorf, S.Z. 1993, Complete Manual of Industrial Safety. (Prentice-Hall). Mehta, P., Sadasivan, S., Greenstein, J.S., Gramopadhye, A.K. & Duchowski, A.T. 2005, Evaluating different display techniques for communicating search strategy training in a collaborative virtual aircraft inspection environment. In Proc. Human Factors and Ergonomics Society 49th Annual Meeting. Ockerman, J., & Pritchett, A. 2000, A review and reappraisal of task guidance: aiding workers in procedure following. Int. J. Cognitive Ergonomics, 4, 191–212. Rasmussen, J. 1986, Information Processing and Human-Machine Interaction: An Approach to Cognitive Engineering, (North Holland). Reich, A.R. 1986, Plant inspection and worker protection. In, LaDou, J., An Introduction to Occupational Health and Safety. (National Safety Council). Technical Standards and Safety Authority. 2006, 2005/2006 Annual Report. (TSSA). Woodcock, K. 1989, Accidents and injuries and ergonomics: A review of theory and practice. In Proc. Human Factors Association of Canada. (pp. 1–10). (HFAC). Woodcock, K. 2004, Individual differences and the use of job aids in the task of amusement ride inspection. Proc. Association of Canadian Ergonomists, 5 pgs.
RECENT ADVANCES IN TECHNOLOGY TO IMPROVE INSPECTION TRAINING A. Gramopadhye, P. Stringfellow & S. Sadasivan Clemson University, Clemson, SC, USA Visual inspection is an important activity and it shows up in various domains – product inspection in the manufacturing, aircraft inspection in aviation, baggage inspection in security, etc. Hence it is vitally important that inspection is performed efficiently and effectively. Training has been identified as the primary intervention strategy to improve the quality and reliability of inspection. If training is to be successful, it is critical that we provide inspectors with appropriate training tools and environment. In response to this need, the paper outlines the effective use of synthetic environments to help train inspectors. The paper focuses on current work being pursued by the Advanced Technology Systems Laboratory at Clemson University, South Carolina in improving the inspection training and delivery process for the aviation industry.
Introduction The aircraft maintenance industry is complex consisting of several interrelated human and machine components. In the aircraft industry, 90% of all inspection is visual and is conducted by human inspectors whose reliability is fundamental to an effective maintenance system. The inspection process essentially consists of two important cognitive components – visual search and decision making. Simply defined, the visual search process involves locating defects in a visual search field and the decision making involves deciding on the severity of the identified defect. If visual inspection performance has to be improved it is critical that both visual search and decision making are performed reliably and consistently. However, human inspection tends to be less than 100% reliable. This human element, however, cannot be entirely eliminated because of our superior decision-making ability (Thapa et al., 1996), our ability to adapt to unforeseen events. As it is important that visual inspection be performed effectively, efficiently, and consistently over time, continuing emphasis has been placed on developing interventions to make inspection more reliable and error tolerant, with training being the strategy of choice for improving the performance of aircraft maintenance inspectors. Researchers (Patrick, 1992; Drury et al., 1992; Gramopadhye et al., 1997) have identified the training content, training methods and delivery system as the important components of a well designed training program. However, training for improving the visual inspection skills is generally lacking at aircraft maintenance facilities even though its impact on improving visual inspection 363
364
A. Gramopadhye, P. Stringfellow & S. Sadasivan
skills has been well documented (Moll, 1980; FAA, 1991; FAA, 1993; Drury et al., 1995). It has been shown to improve the performance of both novice and experienced inspectors (Weiner, 1975; Drury et al., 1990; Gramopadhye et al., 1995). In particular, the success of off-line training/retraining with feedback suggests that this method can play an important role in aircraft inspection training. However, existing training for inspectors in the aircraft maintenance environment tends to be mostly on-the-job training (OJT) (Latorella et al., 1992). This method may not be the best one because feedback, so important to training, may be infrequent, unmethodical, and/or delayed. Moreover, in certain instances feedback is economically prohibitive or impractical because of the nature of the task. Since the benefits of feedback in training have been well documented (Weiner, 1975; Gramopadhye et al., 1997), and for other reasons as well, alternatives to OJT are sought. One of the most viable approaches in the aircraft maintenance environment, given its many constraints and requirements, is computer-based training which offers several advantages over traditional training approaches: it is more efficient while at the same time facilitating standardization and supporting distance learning.
Computer technology applied to training With computer technology becoming cheaper, the future has brought in an increased application of advanced technology to training. Instructional technologists have offered numerous technology-based training devices with the promise of improved efficiency and effectiveness over the past couple of decades. Such training devices are being applied to a variety of technical training applications, including computerbased simulation, interactive videodiscs, and other derivatives of computer-based applications and technology-based delivery systems. In the domain of visual inspection, the earliest efforts to use computers for off-line inspection training were reported by Czaja and Drury (1981), who used keyboard characters to develop a computer simulation of a visual inspection task. Similar simulations have also been used by other researchers to study inspection performance in a laboratory setting. Since these early efforts, Latorella et al. (1992) and Gramopadhye et al. (1993) have used low fidelity inspection simulators with computer-generated images to develop off-line inspection training programs for inspection tasks. More recently, researchers have studied human performance using computer simulations for printed circuit board inspection with great success. Another domain which has seen the application of advanced technology is the inspection of X-rays for medical practice. Most of the relevant work in using technology in training has focused on developing low fidelity simulators for running controlled studies in a laboratory environment. However, research efforts need to be extended in order to take full advantage of the current advances in computer technology. Moreover, advanced technology has found limited application for inspection training in the aircraft maintenance environment. The message is clear: we need more examples of advanced technology applied to inspection training.
Recent advances in technology to improve inspection training
365
The Advanced Technology Systems Laboratory at Clemson University has been at the fore front in the application of computer technology to inspection training to facilitate improvement in inspection performance. This paper outlines two such systems, a PC based low fidelity simulator entitled ASSIST developed in 2000 and a Virtual Reality based high fidelity inspection simulator developed in 2002. They represent an evolution in the application of the use of technology for inspection training.
Automated System of Self-Instruction for Specialized Training The Automated System of Self Instruction for Specialized Training (ASSIST) is a Computer Based Training (CBT) program for use in aviation inspection training, which was developed using task analytic methodology and features a PC-based aircraft inspection simulator (Gramopadhye et al., 2000). The computer-based training program was developed using Visual C++, Visual Basic and Microsoft Access with the development work conducted on a Windowsbased PC platform. The training program uses text, graphics, animation, and audio. The inputs to the system are entered through a keyboard and a mouse. ASSIST has three modules: the General Inspection Module which instructs the trainee on good practices or the right way of doing inspection; the Simulation Training Module (Figure 1) which allow the trainee to perform inspection tasks in various inspection scenarios; and the Instructor’s Utilities Module where an instructor can look at the trainee’s inspection data and also allows the instructor to create different defect scenarios. The results of the follow-up study conducted to evaluate the usefulness and transfer effects of ASSIST were encouraging with respect to the effectiveness of computer-based inspection training, specifically in improving performance (Gramopadhye et al., 1998). Despite their advantages, existing multimedia-based technology solutions, including low fidelity simulators like ASSIST, still lack realism as most of these tools use only two-dimensional sectional images of airframe structures and, therefore, do not provide a holistic view of the airframe structure and the complex maintenance/inspection environment. More importantly, the technicians are not immersed in the environment, and, hence, they do not get the same look and feel of inspecting/maintaining a wide-bodied aircraft. To address these limitations, virtual reality technology.
Virtual reality for training The development of virtual reality (VR) as a tool for inspection training has taken CBT to a new level. (Duchowski et al., 2000). The development of the VR environment was based on a detailed task analytic methodology wherein data on aircraft inspection activity was collected through observation, interviewing, shadowing, and digital data capturing techniques. Using VR trainees can now be fully immersed in the training environment. If VR is to serve as a solution for off-line training, it
366
A. Gramopadhye, P. Stringfellow & S. Sadasivan
Figure 1. ASSIST simulation module.
Figure 2. Aft cargo bay of L1011 aircraft.
Figure 3. Virtual reality model of aircraft Figure 4a. Trainee immersed cargo bay. in VR. is essential that this environment accurately mimic the real world as perceived by the user or trainee (also referred to as the level of “presence” in the VR environment). Only then can the effects of training be expected to transfer from the VR environment to the real world. The scenarios modeled using VR are the aft cargo bay (Figure 2 and Figure 3), wing and fuselage of a wide-bodied aircraft, where pre-generated defects are systematically placed for trainees to detect. The simulated defects were derived from actual defects, these include rivet cracks, holes, corrosion, abrasion, crease and broken conduits. Experiments conducted at Clemson University using VR have been successful in demonstrating that there is improvement in the inspector’s performance following VR training. (Vora et al., 2001) Analysis of the correlation data from the Vora study revealed that the subjects who experienced high involvement in the simulator felt that their experiences were as natural as the real world experiences (i.e., a greater degree of presence).
VR technical specifications The principal hardware component is a head mounted display (HMD) (Figure 4b) integrated with a binocular eye tracker, jointly built by Virtual Research and ISCAN.
Recent advances in technology to improve inspection training
Figure 4b.
Head Mounted Display and 3-D mouse.
367
Figure 4c. Eye tracker integrated within the HMD.
The V8 HMD offers a 640 × 480 pixel resolution for each eye, with separate video feeds for the left and right eye and offers a 75.3◦ × 58.4◦ visual angle. The ISCAN RK-726PCI High Resolution Pupil/Corneal Reflection Processor uses corneal reflections (first Purkinje images) of infra-red LEDs mounted within the helmet (Figure 4c) to measure eye movements. The processor operates at a rate of 60 Hz (30 Hz when both eye movements are tracked) and the subject’s eye position is determined with an accuracy of approximately 0.3 degrees over a 20 degree horizontal and vertical range using the pupil/corneal reflection difference. The maximum spatial resolution of the calculated Point of Regard (POR) provided by the eye tracker is 512 × 512 pixels per eye. Ascension Technology Corporation’s Flock of Birds (FOB) electromagnetic tracking system is used for rendering the virtual scenario with respect to the participant’s position and movements. For the purpose of selection and pointing in the virtual environment, a hand held mouse with six degrees of freedom is used. The simulator is launched on a 1.5 GHz dual Xeon processor Dell personal computer with 1 GB RAM and NVidia 6800 Ultra graphics card running Linux (FC3).
System structure The software component of this simulator consists of three separate programs – Inspector, Vspec and Inspect. Inspector displays the virtual reality scenario to the participants while at the same time recording their eye movements. The Vspec program (Figure 5) is used to analyze the data collected by Inspector. The Inspect program enables the experimenter to create inspection scenarios and affords the creation of a collaborative virtual environment (CVE) which allows the co-immersion of the trainer and the trainee in the virtual the aircraft inspection environment. Figure 6 shows an avatar of the trainee that is used in the collaborative VR environment.
Research studies using VR inspection training Following the development of the VR simulator several controlled studies have been conducted to evaluate the effectiveness of VR training. These studies have
368
A. Gramopadhye, P. Stringfellow & S. Sadasivan
Figure 5. Trainee’s scan path analyzed using Vspec.
Figure 6. Avatar of a trainee in the CVE as viewed by the trainer.
been extensively reported in the open literature. The following paragraphs provide a brief outline of the more salient ones. 1. Presence and performance measurement study (Vora et al., 2001). In this study, participants were taken to the hangar and shown the real aft cargo bay of a wide-bodied aircraft. Then they were taken to the VR lab and immersed in the virtual cargo bay scenario. They performed a task involving walking through the environment and identifying defects present. The results revealed a high level of “presence” experienced by the subjects. Participants indicated that the experiences with the VR environment were consistent with their real world experiences. 2. Feedback Training Using VR (Bowling, 2008): This study evaluated the effectiveness of alternate feedback strategies on visual search performance. Here participants were asked to perform an inspection task in the virtual environment. Performance measures and process measures were recorded and then they were provided as feedback training followed by an inspection task. The results showed an improvement in inspection speed and accuracy following training. 3. Feed forward Training using VR (Bowling, 2008): This study focused on the effectiveness of using prior information on training. Prior information that included knowledge about defect characteristics – types, severity/criticality, location, and the probability of occurrence was provided to the participants as feed forward information. Subjects who were trained on using prior information showed better performance on both the visual search and decision making components of the inspection task following feed forward training. 4. Search Strategy Training: This study focused on training novice inspectors to adopt an expert’s search strategy (Sadasivan et al., 2005). Eye tracking data was used to capture an expert’s search strategy which was later used to train novice inspectors. Analysis revealed that inspectors could use this information to adapt to a new search strategy which in turn led to improvements in inspection performance.
Recent advances in technology to improve inspection training
Figure 7.
Enhanced VR cargo bay model.
369
Figure 8. Video borescope.
5. Collaborative Virtual Environments were evaluated for providing simulated on-the-job search strategy inspection training and were found to be significantly better than just practice in improving inspection performance (Sadasivan et al., 2005a,b; Mehta et al., 2005).
Conclusions and extensions Following the success in using VR for inspection training, current research has focused on developing a high-impact, hands-on virtual reality model of the aircraft inspection and maintenance process for curriculum application in aircraft maintenance technology schools. As a part of this the prototype VR simulator is being enhanced (Figure 7) using 3D modeling techniques and lower fidelity implementations (Sadasivan et al., 2007, 2006a). This use of virtual reality technology will enable educators to create and students to experience the complex aircraft maintenance environment in an educational classroom setting. The successful completion of this effort will fill an industry need for well-prepared students entering the aircraft maintenance industry, and will lead to a better understanding of the use of virtual reality as a pedagogical tool to improve the instructional process. Beyond direct visual inspection, the aircraft maintenance industry also uses Non-destructive Inspection (NDI) techniques. The Advanced Technology Systems Laboratory is currently focused on creating VR models of training in support of these different NDI techniques. One example, is the video borescope (Figure 8) used to conduct remote visual inspection of difficult to reach aircraft engine parts; consisting of a hand-held device comprised of a video screen and a long flexible probe with a camera attached to its tip. The virtual borescope simulator (Figure 9, 10, 11) being developed at Clemson seeks to mimic this inspection procedure and the details of this research can be found in Vembar et al. (2005, 2008) and Sadasivan et al. (2006b). Through the development and systematic application of human factors techniques, the current research in inspection training has focused on improving the effectiveness and efficiency of aircraft visual inspection by integrating training
370
A. Gramopadhye, P. Stringfellow & S. Sadasivan
Figure 9. Actual (left) and modeled (right) engine sections.
Figure 10.
Basic gamepad based prototype (left) and simulator implementation integrated with probe feed and haptics (right).
Figure 11.
Comparison visual output from the actual borescope (left) and VR simulator (right).
methods that work with state of the art delivery system. The results of these efforts are now available to the aviation maintenance community to improve inspection. The future holds great promise to extend these gains to other domains such as security, medicine and manufacturing.
Recent advances in technology to improve inspection training
371
References Bowling, B., Khasawneh, S.R., Kaewkuekool, M. T., Jiang, X., and Gramopadhye, A. K., 2008, Evaluating the Effects of Virtual Training in an Aircraft Maintenance Task, International Journal of Aviation Psychology, 18:1, 104–116. Czaja, S.J., Drury, C.G., 1981, Training programs for inspection. Human Factors 23 (4), 473–484. Drury, C.G. and Gramopadhye, A.K., 1990, Training for visual search. Proceedings of the 3rd FAA Meeting on Human Factors in Aircraft Maintenance and Inspection: Training Issues. Drury, C.G., 1992, Inspection performance, in Handbook of Industrial Engineering, 2nd ed. by G. Salvendy (ed.) Wiley and Sons, New York. Drury, C.G. and Chi, C.F., 1995, A test of economic models of stopping policy in visual search. IIE Transactions, 382–93. Duchowski, A.T., Shivashankaraiah, V., Rawls, T., Gramopadhye, A.K., Melloy, B., Kanki, B., 2000, Binocular eye tracking in virtual reality for inspection training. In Proceedings of the Eye Tracking Research & Applications Symposium, Palm Beach Gardens, FL, November 6–8, 89–96. Federal Aviation Administration (FAA), 1991, Human Factors in Aviation Maintenance Phase One Progress Report, DOT/FAA/AM-91/16 (Washington DC: Office of Aviation Medicine). Federal Aviation Administration (FAA), 1993, Human factors in aviation maintenance. Phase Two Progress Report, DOT/FAA/AM-93/5. Office of Aviation Medicine, Washington, DC. Gramopadhye, A.K., Drury, C.G., and Sharit, J., 1993, Training for decision making in aircraft inspection. Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting, pp. 1267–71. Gramopadhye, A.K., Ivaturi, S., Blackmon, R.B., Kraus, D.C., 1995, Teams and teamwork:implications for team training within the aircraft inspection and maintenance environment. FAA-1995 Technical Report. Office of Aviation Medicine, FAA. Gramopadhye, A.K., Drury, C.G., and Prabhu, P.V. 1997, Training strategies for visual inspection, Human Factors and Ergonomics in Manufacturing, 7(3), 171–196. Gramopadhye, A.K., Bhagawat, S., Kimbler, D., Greenstein, J. 1998, The use of advanced technology for visual inspection training. Applied Ergonomics 29(5), 361–375. Gramopadhye, A.K., Melloy, B.J., Chen, S., Bingham, J., 2000, Use of computer based training for aircraft inspectors: findings and recommendations. In Proceedings of the HFES/IEA Annual Meeting, August ’00, San Diego, CA. Latorella, K.A., Gramopadhye, A.K., Prabhu, P.V., Drury, C.G., Smith, M.A., and Shanahan, D.E., 1992, Computer-simulated aircraft inspection tasks for offline experimentation. Proceedings of the Human Factors Society 36th Annual Meeting, pp. 92–96. Mehta, P., Sadasivan, S., Greenstein, J., Duchowski, A., and Gramopadhye, A., 2005, Evaluating Different Display Techniques for Search Strategy Training in a
372
A. Gramopadhye, P. Stringfellow & S. Sadasivan
Collaborative Virtual Aircraft Inspection Environment. In proceedings of Human Factors and Ergonomics Society Annual Conference, Orlando, FL. Moll, R.A., 1980, Product liability:a look at the law. Eng. Educ. 66, 326–331. Patrick, J., 1992, Training Research and Practice. New York: Academic Press. Sadasivan, S., Vembar, D., Washburn, C., and Gramopadhye, A., 2007, Evaluation of Interaction Devices for Projector Based Virtual Reality Aircraft Inspection Training Environments. In proceedings of HCI International Annual Conference, 22–27 July, Beijing, P.R.China. Sadasivan, S., Vembar, D., Stringfellow, P.F., Gramopadhye, A.K., 2006b, Evaluation of Interaction Devices for NDI Training in VR: Gamepad vs. Joystick, In proceedings of Human Factors and Ergonomics Society Annual Conference, San Francisco, CA. Sadasivan, S., Vembar, D., Stringfellow, P.F., Gramopadhye, A.K., 2006a, Comparison of Window-VR and HMD Display Techniques in Virtual Reality Inspection Environments, In Proceedings of the International Ergonomics Association Conference, July 10–15, Maastricht, Netherlands. Sadasivan, S., Rele, R. S., Masters, J., Greenstein, J., Duchowski, A., and Gramopadhye, A., 2005b, Collaborative Virtual Environment to Simulate OnThe-Job Aircraft Inspection Training Aided by Hand Pointing. In proceedings of Human Factors and Ergonomics Society Annual Conference, Orlando, FL. Sadasivan, S., Rele, R. S., Greenstein, J., Duchowski, A., and Gramopadhye, A., 2005a, Simulating On-the-Job Training using a Collaborative Virtual Environment with Head Slaved Visual Deictic Reference. In proceedings of HCI International Annual Conference, 22–27 July, Las Vegas, NV. Sadasivan, S., Greenstein, J. S., Gramopadhye, A. K., and Duchowski, A. T., 2005, Use of Eye Movements as Feedforward Training for a Synthetic Aircraft Inspection Task, CHI ’05, April 2–7, 2005, Portland, OR, ACM. Thapa, V.B., Gramopadhye, A.K., Melloy, B., Grimes, L., 1996. Evaluation of different training strategies to improve decision making performance in inspection. Int. J. Human Factors Manuf. 6 (3), 243–261. Vembar, D., Duchowski, A., Sadasivan, S., and Gramopadhye, A., 2008, A Haptic Virtual Borescope for Visual Engine Inspection Training. In IEEE Symposium on 3D User Interfaces (3DUI), March 8–9, Reno, NV, IEEE. Vembar, D., Sadasivan, S., Duchowski, A.T., Stringfellow, P.F., and Gramopadhye, A., 2005, Design of a Virtual Borescope: A Presence Study, HCI International, 22–27 July, Las Vegas, NV. Vora, J., Nair, S., Medlin, E., Gramopadhye, A. K., Duchowski, A. T., & Melloy, B. J., 2001, Using virtual reality to improve aircraft inspection performance: Presence and Performance Measurement Studies. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Minneapolis, MN. Weiner, E.L., 1975, Individual and group differences in inspection. In Drury and Fox (Editors) Human Reliability in Quality Control (London: Taylor & Francis).
USING BODY DISCOMFORT CORRELATIONS TO ASSIST IN ERGONOMIC INSPECTIONS OF COMPUTER USERS P. Stringfellow & A. Gramopadhye Clemson University, Clemson, SC, USA Recent findings highlighting the link between ergonomic factors and injury in the workplace have resulted in increased ergonomic awareness in many industries. Most often, ergonomic evaluators and evaluation practitioners use their knowledge and experience to audit and improve the ergonomic quality of a workplace environment. These inspections are frequently based on industry-specific recommendations or “best practices,” but can easily miss important issues that may not be evident using such a methodology. The process of ergonomic auditing can be considered highly analogous to many of the processes that are classified under visual inspection, as the ergonomist performs a scan of the environment to detect a specific feature among other distracters or features. This paper presents a methodology and framework for applying the principles of systematic visual inspection and ergonomic event relationships to traditional ergonomic inspection and evaluation.
Introduction The modern office is equipped with a variety of technologies from wireless printers to hand-held PDA’s, the most prominent of which is the standard desktop PC. A recent US census revealed that seventy-six million employed US adults use a computer at their place of work (Cheeseman, et al., 2003), an increase of 17% since the last census taken in 2001, and reports indicate that this trend is steadily growing. This piece of technology has single handedly redefined the traditional work environment where once paperwork, couriers, dictation specialists, design engineers (blue print drafting) each performing completely different work-related tasks are now confined to the parameters of a 4 × 5 workspace, a mouse, a keyboard, and a monitor. With this increase in computer use, so too has arisen a new strain of Workrelated Musculoskeletal Disorders (WMSD’s). In 1999, Liberty Mutual reported that hand, wrist and shoulder injuries were a fast growing source of disability in the American workforce rising in large part from the increase in computer usage in the workplace (Liberty Mutual, 1999). Musculoskeletal pain and discomfort, together with eyestrain, make up at least half of all subjective complaints of computer operators (Helander, 1988). Each year estimated costs resulting from computer-based WMSD’s range in the millions, while employees are suffering from lost pay, and sometimes lifelong afflictions. Fortunately, the potential impacts of extended computer use are being realized and much work and research has been done related to the ergonomics of computer workstations.
373
374
P. Stringfellow & A. Gramopadhye
Designing for the office place requires the consideration of a number of different elements, from job rotation schedules to choosing the right chair. It can be broken up into four subsequent areas of focus: the organizational structure, environmental factors, the individual workstation, and the individual worker (WISHA, 2002). The organizational component looks at job design, staffing levels and procedures, as well as work schedules as they impact the worker. Here, items that are controlled through administrative procedures, such as scheduled break times are considered. Within the organizational structure of the workplace, the office environment plays a part in affecting the ergonomic quality of the worker. Environmental elements such as monitor glare from uncovered windows or excessive noise from a busy hallway can influence both worker productivity and comfort. When environmental conditions cannot be controlled universally, personal adjustments can be made to the individual workplace. At this level, elements such as furniture and accessories can be added and customized to the worker’s personal needs. Although much can be done to enhance the organization, environment, and workspace surrounding the worker, the worker’s behavior toward ergonomics can also be improved through training and experience. Risk factors to the development of WMSD’s include physical, psychosocial and organizational, and individual elements. Within the VDT workstation, physical factors include appropriate postural positioning and loading, minimizing the existence of pressure points, maintaining adequate clearance between the body and the workstation, and the mitigating the negative effects of the environment (eg. noise, temperature, lighting, etc.). A number of methods exist for assessing the exposure to risk factors for workrelated musculoskeletal disorders (David, 2005). Among other methods, subjective body discomfort assessment tools provide a direct means of capturing operator effect from workstation design parameters. Specifically, discomfort measures have been used to identify sources of ergonomic strain within a work area, prioritize the importance of ergonomic interventions (Marley and Kumar, 1994) and to assess the effects of varying workstation parameters on operator musculoskeletal discomfort (Sauter et al., 1991). Although a collection of discomfort (and comfort) assessment techniques are available (Cameron, 1996), the system originally developed by Corlett and Bishop (1976) provides a concise and easily distributed data collection tool for use in industrial settings. Past research has sufficiently looked at the correlation of ergonomic parameters with body discomfort. Regarding VDT work, the ergonomic parameters postural positioning (Liao and Drury, 2000), postural loading (Boussenna et al., 1982), pressure points (Mehta and Tewari, 2000), and clearance (Hira, 1980) have all shown correlation with body discomfort. However, only piecemeal work has been done when considering the predictive relationships between ergonomic parameters themselves. That is, in VDT work, certain postural positions may directly affect the existence and severity of certain pressure points, for example. When considering the role of visual inspection in an ergonomic context, most visual search research has focused on the application of visual search to ergonomics in the traditional sense, often with the objective of modifying work requirements or equipment layout. However, theory of visual search can be applied to the process of
Using body discomfort correlations to assist in ergonomic inspections
375
ergonomic inspection as well. Common practice for ergonomic inspections, particularly in a VDT workstation setting, quite often relies on the industrial ergonomist’s experience and integrated understanding of the workstation-operator system as a whole to correctly and efficiently identify and correct issues. Often, checklists and other aids are used to assist the inspector in his/her identification of the issues. Aids such as these, help to ensure a systematic search strategy, thus reducing the probability for potential misses. As a result, however, the ergonomist spends excessive and potentially unnecessary time considering noise or items that may have a low probability of occurring given the current ergonomic conditions. Thus, the ergonomic inspection system is one which is dynamic in nature and which could benefit from feed-forward information as to the relativity of given ergonomic parameters. The concept of modeling musculoskeletal disorder risk associated with occupational exposures is not new, and other researchers have well demonstrated applications of advanced statistical methods to predict such risk (Chen et al., 1999; Gilad and Karni, 1999; Dempsey, et al., 1998; Zurada et al., 1997; Karwowski et al., 1994; Marras et al., 1993). The work presented here contends that WMSD risks, signified by worker body part discomfort, are context dependent and vary when associated with different occupational settings. This paper presents the beginnings of a system to serve as a dynamic ergonomic inspection aid based on the inherent relationships of ergonomic parameters in a VDT workstation. Initial correlations for these relationships were derived from body discomfort data gathered from an industrial setting. The next section discusses the collection and correlation of this data, while the following section presents a framework for incorporating these findings into a predictive model.
Ergonomics at a computer workstation: Baseline study In an effort to quantitatively identify related ergonomic events at a computer workstation, a study was conducted to evaluate the ergonomic quality of a large organization where computer work was a primary task. In this case, ergonomic quality was considered a function of subjective body discomfort as reported by the study participants.
Participants The study evaluated a total of 172 computer operators, 25% male and 75% female, whose mean age was 46.5 years (S.D. = 12.2 years). On average, participants spent 24.1 hours per week (S.D. = 10.8 hrs./wk.) directly operating their computer at work.
The site The study looked at computer operators located at 10 different worksites throughout the U.S. The organization maintained strict adherence to uniform workstations using standardized equipment and facilities, so that minimum variation was introduced
376
P. Stringfellow & A. Gramopadhye
due to location differences. All worksites were equipped with a panel-mounted modular furniture system to create cubicles for the workstations. The panels were sound retardant, the furniture had rounded corners, separate cubicle lighting was provided and the work surface heights were vertically adjustable. All study participants had adjustable seats. Other ergonomic-enhancing aids, such as foot rests, additional lumbar support, and keyboard wrist rests, were available and utilized at the discretion of the participants.
Procedure An ergonomic evaluation of each workstation was conducted. Workstations were evaluated for posture, static load, pressure points, clearance, and environmental factors (eg. lighting, noise, etc.). Baseline body discomfort data was also collected using the 5-point body discomfort questionnaire developed by Corlett and Bishop (1976), which is anchored at the 0 point with ‘no discomfort’, and at the 5 point with ‘extreme discomfort.’ The survey was administered to each participant at the beginning and the end of their standard workday prior to the implementation of any ergonomic interventions.
Results To account for individual biases, the difference between the beginning-of-workday and end-of-workday was used in the analysis. Summary statistics for the discomfort data is shown in Table 1. As the data was measured on an ordinal scale, a correlation analysis using a 1-tailed Spearman’s Rank Test (α = .05) was conducted on the difference discomfort data and the results are shown in Table 2. All findings were significant using α = 0.05. Also, where Table 2 is truncated, a medium correlation exists. Table 1.
Spearman’s rank correlation of body discomfort data. Min
Max
Mean
Std. Dev.
Min
Max
0
4
0.9186
1.25856
Right Foot
0
3
0.1337
0.5294
Front Right Shoulder 0
5
0.5349
1.0673
Left Foot
0
3
0.1512
0.62159
Front Left Shoulder
0
5
0.407
0.98382
BackNeck
0
5
1.0174
1.28178
Right Arm
0
4
0.3488
0.86203
Back Right Shoulder
0
5
0.6512
1.14726
Left Arm
0
4
0.2326
0.72828
Back Left Shoulder
0
5
0.5174
1.06773
Abdomen
0
4
0.0988
0.47952
UpperBack
0
5
0.7093
1.17842
Right Wrist
0
5
0.5
1
MidBack
0
5
0.6047
1.14738
Left Wrist
0
3
0.2965
0.75656
LowerBack
0
5
0.8547
1.27359
Right Hand
0
5
0.4419
0.96251
Buttocks
0
5
0.3256
0.82988
Left Hand
0
4
0.2674
0.76356
Back-Right Leg
0
4
0.1977
0.62725
Right Knee
0
4
0.2093
0.6599
Back-Left Leg
0
4
0.2151
0.69686
Left Knee
0
4
0.157
0.5961
Right Calf
0
3
0.1512
0.51905
Right Ankle
0
3
0.1395
0.54425
Left Calf
0
4
0.1453
0.54808
Left Ankle
0
3
0.0872
0.44388
Eye
Mean
Std. Dev.
N = 172 for all variables
Using body discomfort correlations to assist in ergonomic inspections
377
Discussion The data revealed some expected results. Overall, the summary results were consistent with previous findings where eye fatigue and neck pain were paramount factors (Iwakiri, et al., 2004) Primarily, weak correlations were found between body parts regionally distant from each other. For example, there appeared to be no correlation between the right knee and the left arm (ρ = 0.200, p-value ≤ 0.05). Conversely, stronger correlations existed between like body parts, such as the left and right calves (ρ = 0.791, p-value ≤ 0.05) and the right shoulder and the neck (ρ = 0.705, p-value ≤ 0.05). It can be projected then, that as the risk of discomfort increases on certain parts of the body, so too does that risk increase on other highly correlated parts. Table 2.
Spearman’s rank correlation of body discomfort data.
378
P. Stringfellow & A. Gramopadhye
A model for office ergonomic risk assessment Given the evident correlation in discomfort among body parts when performing VDT work, it can then be conjectured that the relative risk factors are also related. For example, pain in the neck is highly correlated with pain in both the upper back and the right shoulder. The cause of this discomfort could be due to either postural effects, such as a bent neck, or to static loads on the operator, such as unsupported arms. Hence, if a bent neck is observed during an inspection, the evaluator should also inspect for arm support. The figure below illustrates the relationships (only strong correlations shown) between VDT risk factors and associated body parts. As suggested in work by Gramopadhye et al. (1997), inspection training coupled with feedforward inspection information can assist in modifying search strategy and ultimately help to improve mean search times. It is anticipated that the full development of a model such as the one proposed above will aid the trained ergonomic evaluator during routine inspection of VDT workstations for outstanding ergonomic discrepancies by providing him or her with feedforward data as to potential risk areas. Further, the approach adopted in this research begins by considering body discomfort as a driver rather than the correlations directly associated with the respective VDT risk factors themselves. While this may at first appear contrary or “working from the bottom up” the approach was selected as it relates directly to the process which typically drives ergonomic evaluations in the workplace, where operators first report body discomfort as a symptom of some underlying ergonomic discontinuity. In effect, this approach intended to take the reactionary nature of standard ergonomic inspection and use it to develop a system which is then proactive in nature, so as to preempt the causes which invoke the reaction in the first place.
Figure 1.
Office ergonomic event correlation diagram.
Using body discomfort correlations to assist in ergonomic inspections
379
Future work While the context of this work is specific to the application of office ergonomics, the process has potential for applications in other areas as well, such as manufacturing, aerospace, health care and other well-defined work environments. In particular, this model is best suited to work in static, well-defined and highly repetitive environments. Furthermore, it should be noted that the correlations identified here are not universal, but are specific to the work type discussed. Next steps in this research are to verify the risk factor correlation matrix empirically in an office workstation environment and using an ergonomic simulation analysis tool. The eventual results of this work will aid ergonomic inspectors by providing them with an adaptive tool for evaluating and improving the ergonomic quality of an industrial workspace.
References Boussenna M., Corlett E.N., Pheasant S.T. 1982, The relation between discomfort and postural loading at the joints, Ergonomics, 25(4): 315–322. Cameron, J.A., 1996, Assessing work-related body-part discomfort: Current strategies and a behaviorally oriented assessment tool, International Journal of Industrial Ergonomics Selected Papers from the 10th Annual International Industrial Ergonomics and Safety Conference, 18 (5–6): 389–398. Cheeseman, J.D., Janus, A., Davis, J. (U.S. Census of Population and Housing), Computer and Internet Use in the United States: 2003, Special Studies, Government Printing Office, October 2003. Chen, C.-L., Kaber, D.B., Dempsey, P.G. 2000, A new approach to applying feedforward neural networks to the prediction of musculoskeletal disorder risk, Applied Ergonomics, 31(3): 269–282. Corlett, E. N. & Bishop, R. P. 1976, A technique for measuring postural discomfort. Ergonomics, 9, 175–182. David, G.C. 2005, Ergonomic methods for assessing exposure to risk. Occupational Medicine, 55: 190–199. Dempsey, P.G., 1998. A critical review of biomechanical, epidemiological, physiological and psychophysical criteria for designing manual materials handling tasks. Ergonomics 41, 73–88. Helander, M.G. (ed.). 1988, Handbook of Human Computer Interaction. Amsterdam: North-Holland. Hira, D.S. 1980, An ergonomic appraisal of educational desks, Ergonomics, 23(3): 213–221. Liao, M.-H., Drury, C.G., 2000. Posture, discomfort, and performance in a VDT task. Ergonomics, 43 (3), 345–359. Iwakiri K, Mori I, Sotoyama M, Horiguchi K, Ochiai T, Jonai H, Saito S. 2004, Survey on visual and musculoskeletal symptoms in VDT workers, Journal of Occupational Health [Japan], 46(6): 201–212.
380
P. Stringfellow & A. Gramopadhye
Karwowski, W., Zurada, J., Marras, W.S., Gaddie, P., 1994. A prototype of the artificial neural network-based system for classification of industrial jobs with respect to risk of low back disorders. In: Aghazadeh, F. (Ed.), Advances in Industrial Ergonomics and Safety VI Taylor & Francis, Bristol, PA, pp. 19–22 Liberty Mutual Research Center for Safety and Health, 1999, From research to reality. (Annual Report). Hopkinton, MA: Liberty Mutual Research Center for Safety and Health, 14. Gilad, I., and Karni, R. 1999, Architecture of an expert system for ergonomics analysis and design, International Journal of Industrial Ergonomics, 23, 205–221. Gramopadhye, A.K., Drury, C.G., and Prabhu, P.V. 1997, Training strategies for visual inspection. Human Factors and Ergonomics in Manufacturing, 7(3), 171–196. Marley, R.J. and Kumar, N., 1994. An improved musculoskeletal discomfort assessment tool. In: F. Aghazadeh (Ed.), Advances in Industrial Ergonomics and Safety VI. Taylor and Francis, London, pp. 45–52. Marras, W.S., Lavender, S.A., Leurgans, S., Sudhakar, L.R., Allread, W.G., Fathallah, F., Ferguson, S., 1993, The role of dynamic threedimensional trunk motion in occupationally-related low back disorders. Spine 18, 617–628. Mehta, C.R. and Tewari, V.K. 2000, Seating discomfort for tractor operators – a critical review, International Journal of Industrial Ergonomics, 25(6): 661–674. Sauter, S.L., Schliefer. L.M. and Knutson, S.J., 1991. Work posture, workstation design, and musculoskeletal discomfort in a VDT data entry task. Human Factors, 33(2): 151–167. WISHA Services Division, 2002, Office Ergonomics: Practical Solutions for a Safer Workplace, Washington State Department of Labor and Industries. Zurada, J. Karwowski, W., Marras, W.S., 1997, A neural network-based system for classification of industrial jobs with respect to risk of low back disorders due to workplace design. Applied Ergonomics, 28(1), 49–58.
ONE YEAR LATER: HOW SCREENER PERFORMANCE IMPROVES IN X-RAY LUGGAGE SEARCH WITH COMPUTER-BASED TRAINING A. Schwaninger & A.W.J. Wales University of Zurich, Zurich, Switzerland A total of 273 luggage search operators from eight airports performed an x-ray competency assessment test on two occasions, with a median of 427 days apart (s.d. = 172.06 days). These screeners are part of an employment setup where recurrent computer-based training and on-the-job assessment are in action, resulting in a large performance ability improvement between the two tests. Using Drury’s inspection model (1975) accompanied by signal detection measures (Green and Swets, 1966), it was found that screeners became faster for both search and decision time, as well as becoming more accurate in their detection of threat items. It is concluded that longitudinal benefits are found when screeners are part of an organised training environment that is scientifically based and individually adaptive.
Introduction Airport baggage security screeners are increasingly being trained with sophisticated software such as X-Ray Tutor that individually adapts its difficulty to compensate for their improvements in detection performance (Schwaninger, 2004b; Koller et al., 2008; Michel et al., 2007; Schwaninger et al., 2008). At several airports, screeners are continually being tested on-the-job through a system known as “threat image projection” (TIP; e.g. Schwaninger, 2004a; Hofer and Schwaninger, 2005) that superimposes fictional threat items into actual x-ray luggage images. The basic principles behind the TIP system actually predate airport security screening, where Broadbent (1964) notes “it has often been suggested that in practical situations the efficiency of radar monitors or industrial inspectors could be improved by the insertion of artificial signals interspersed amongst the real ones” (p. 18). With modern advances in computer systems (Schaller, 1997) it is now readily possible to examine the performance of screeners while they work and provide a system of certification relating to performance ability. Having competent workers is important in any industry, but particularly so in airport security. Since certain visual abilities are essential to become a good x-ray screener (Hardmeier, Hofer and Schwaninger, 2005; Hardmeier & Schwaninger, 2008), pre-employment testing needs to form a staple part of screener employment procedures. Training that is individually adaptive has the benefit of seeking weaknesses and addressing them with increased load into those areas (Schwaninger, 381
382
A. Schwaninger & A.W.J. Wales
2004b). While screeners need to demonstrate improvements they also need to fall within accepted working norms which can be deduced from retrospective analyses of TIP data and standalone performance testing. As has been shown in previous research (i.e. Hofer and Schwaninger, 2005), there are wide variations in operator ability but with appropriate training the entire distribution should move as each screener makes their own improvements (Schwaninger, Hofer, & Wetter, 2007). Given that terrorist threats are “both productive and diverse” (Lui et al., 2007, p.301), training systems must be adaptive, recurrent and attempt to predict the mindset of potential terrorists in a changeable climate. Security screening is therefore both demanding and important, requiring research-led training systems capable of quantifying real-life performance. Detection performance in terms of sensitivity is typically viewed as being a function of hits (correctly identifying an object that contains a threat item), false alarms (incorrectly stating a threat item is present), and in certain cases confidence ratings (Green & Swets, 1966; MacMillan & Creelman, 1991). A vital aspect of performance is the speed in which an operator can perform bag searches while maintaining optimal performance levels, which signal detection theory cannot accommodate as it assumes a fixed sampling interval (Smith, 2000). Thus, Drury’s Two-Component Model (TCM; Drury, 1975) can be used to approximate the speed taken to search for items, and the average decision time used. This model is a useful complement to signal detection theory and provides insight into whether search or decision time improves with on-the-job training. With the goal of applying SDT and TCM estimates to a large dataset, we have collated the data from a longitudinal project to evaluate performance changes in screeners who are part of a battery of training systems developed by the VICOREG research group (Schwaninger, 2004a, 2004b; Schwaninger, Hofer and Wetter, 2007). While it is impossible to eliminate effects generated by work days during the year, confounding variables or other sources of nuisance variance the advantages of being able to gauge actual real-life performance in screeners far outweigh the detractions. The benefits of utilising scientifically based training can therefore be deduced by comparing the two test performances. With a median date interval exceeding one year between the two tests it can be seen whether screener performance changes using a large sample of workers in a standardised test.
Methods Screeners And Task 273 screeners from eight airports performed the X-Ray Competency Assessment Test (X-Ray CAT, Koller et al. 2008) on two separate occasions. The X-Ray CAT is a simulated luggage search task that presents screeners with images of bags of various difficulties with threat items placed at a base rate of 50%. Subjects only have two choices to make (“ok” or “not ok”), and the images are presented onscreen for up to four seconds. Threat categories include variants of knives, guns, improvised explosive devices and other threats such as tazers or sprays.
One year later: How screener performance improves in x-ray luggage search
Table 1.
A’ B” STh STfa NSTh NSTfa
383
Inferential statistics and mean differences (standard deviations). Test 1
Test 2
Significance Within-Groups
0.793 (0.08) 0.290 (0.27) 0.362 (0.19) 0.315 (0.15) 2.206 (1.13) 3.668 (1.87)
0.864 (0.06) 0.118 (0.37) 0.567 (0.38) 0.396 (0.25) 1.249 (0.55) 2.603 (1.39)
t(273) = −11.667, p < 0.05, d = 0.75 t(273) = 0.818, p > 0.05, d = 0.03 t(273) = −9.343, p < 0.05, d = 0.56 t(269) = −5.059, p < 0.05, d = 0.33 t(273) = 13.362, p < 0.05, d = 0.81 t(269) = 8.215, p < 0.05, d = 0.50
Analyses As subjects were selected based on whether they had performed the test on multiple occasions, a within-subjects analysis can be performed that partials out variance caused by inter-subjects factors (Landauer et al., 2008). This is important because of the lack of experimental control that is found when making deductions based on data that is remotely collected through an autonomous computer network. Several dependent variables can be gleaned from simply evaluating data by subject, reaction time and response. Detection performance, as measured by A’ (Pollock and Norman, 1964) and subjective bias (B”; Grier, 1971) are two readily-prepared statistics that are analogues to those used in signal detection theory (Green and Swets, 1966). These statistics provide an overview of performance irrespective of time and makes less assumptions about the distributions of the data than their signal-detection counterparts. All performance metrics have been multiplied by an arbitrary constant to protect security-sensitive data. Further to this, an approximation of time taken to search for potential threat items and decision time can be made from these three parameters listed above (Drury, 1994). Several papers in the field of aviation have provided complimentary evidence to support Drury’s model when applied to x-ray luggage search data (i.e. Ghylin et al., 2006). In brief, Drury’s model allows estimation of search (STh for hits and STfa for false alarms) and non-search time (NSTh for hits and NSTfa for false alarms), the latter of which is largely made up of decision time, by stepwise linear correlation of performance as judged by Drury’s p(hit/fa) formulation to reaction time by using the model’s formulas. Effectively, the quickest reaction time in a trial tends to become the determinant of pure non-search time as it is assumed to mostly be free of the decision component. This process is calculated for each individual and each CAT test, and then averaged to provide overall model parameters (Table 1).
Results As an indicator of ability, screeners are rated on a scale from 0–12 based on previous experience with adaptive training systems (Michel et al., 2008). Before conducting the X-Ray CAT for the first time, their average performance level was at level 0 (s.d. = 0.15), but by the time screeners had begun their second tests, they had risen to level 8 on average (s.d. = 3.65). The mean date difference (mean = 386.03 days,
384
A. Schwaninger & A.W.J. Wales
Figure 1.
Percentage change in A by test date difference.
s.d. = 172.06 days) between testing days masks the actuality of the performance date differences, as 215 subjects performed the tests more than 300 days apart while 15 screeners took the test less than a month apart resulting in a wide variance (see Figure 1). Therefore the median date (427 days, s.d. = 172.06; range = 5 − 802) provides a more reliable estimate. Only screeners who had performed the test twice were chosen for analysis, thereby eliminating 244 candidates. Figure 1 indicates that the subset of screeners who performed the test less than a month apart actually tended to decrease in their performance, whereas the vast majority of screeners who took the test after more than a month’s interval performed better than on their first occasion. After 600 days, there is a trend towards diminished performance, but a lack of cases make this inference tenuous. Screeners also get progressively faster for correct decisions for threat and non-threat items, as shown by the scatterplot in Figure 2. Correlations between hit reaction time and probability of a hit were low for the first test (r = 0.10, p > 0.05) but statistically significant for the second test (r = 0.30, p < 0.05), while correlations between correct rejection reaction time and probability of a false alarm were high for the first test (r = 0.38, p < 0.05) but lower for the second test (r = 0.14, p < 0.05). Paired-samples t-tests were performed for each metric and effect sizes were generated according to Cohen (1988) between the two tests. Table 1 shows that every performance indicator achieved experimental significance except for bias, which remained highly insignificant. Figure 3 shows histograms for each performance indicator as measured by pooled z-score values so that the effect sizes of different scales can be directly compared. R2 values, separated by pipes, indicate the adjusted model fit for the first test and second test respectively. All tests were found to follow
One year later: How screener performance improves in x-ray luggage search
Figure 2.
385
Stopping time policy for hit RT and Correct rejection RT.
an approximately normal distribution, or more specifically a Gaussian exponential curve, with high r values. Although high, the r2 values indicate that some discrepancies between the actual values and the models exist. By means of comparison, all curves should have the same area underneath them were r to equal 1, but the largest differences are between NSTh 1st and 2nd sessions (165.3 units2 to 125.0 units2 ) and the smallest between STfa 1st and 2nd sessions (147.97 units2 to 139.00 units2 ). With practice and on-the-job experience, the second tests all show greater kurtosis than the first tests for non-search time and detection, but not for search time. This manifests itself as lower standard deviations (see Table 1 and shown in Figure 3), indicating that a ceiling effect approaches with performance tending towards perfection with practice for non-search time and detection performance, although notably never achieved. All graphs are shown on the same frequency scale and z-score distances are all directly comparable. As expected, the greatest performance effects are seen from those screeners who were poorest in their first tests leading towards the distribution shift evident in Figure 3. Negative t-values in Table 1 show as a directional change in Figure 3, where a decrease in value is desirable (i.e non-search time) in some metrics and an increase in others (i.e. A’). Although the coefficient of search-time becomes larger with practice this is a desirable outcome as it indicates improved performance when taken as a unit of change in Drury’s overall model equation.
Discussion A substantial improvement in detection performance is seen in absolute terms (Table 1) and relative terms (Figure 1), as well as improvements in response
386
A. Schwaninger & A.W.J. Wales
Figure 3. Z-Score distributional changes between tests 1 and 2. R2 values indicate the percentage of variance that each model explains for tests 1 and 2 respectively. A , B , NSTh and NSTfa (Non-Search Time for hits and false alarms), STh and STfa (Search Time for hits and false alarms) are given as well as estimates of effect size (d).
time (Figure 2), search time and non-search time (Figure 3) with training. Of all the pairwise comparisons made, only bias failed to achieve experimental significance (Table 1), although as a standalone measure bias has no explanatory worth pertaining to performance. Distributional changes using standardised scores make various dependent variable alterations directly comparable (Figure 3), with accompanying effect sizes providing an empirical determinant of treatment magnitude. If the aim of systemic training is to shift the distributions of performance then the enhanced kurtosis indicated by reduced standard deviations and abscissa drift provides strong evidence that performance is progressively tending towards operator performance limits for
One year later: How screener performance improves in x-ray luggage search
387
detection performance and non-search time. Although search time performance did improve the increased standard deviations indicate that more training would be required to bring all the screeners up to the same standard as a large number did not improve to the same extent that others did. The biggest improvements are seen for hits rather than false alarms, indicating that screeners become more adept at spotting threat items in luggage than merely correctly identifying non-threat items that can sometimes be the explanation for A improvements, which is not explicitly split into threat and non-threat items. As there is no change in bias it cannot be concluded that what changes with training is a tendency to click “bag not ok,” which is a strategy some screeners employ thinking that it will improve their detection results. Table 1 and Figure 3 show in conjunction that there are wide differences in screener ability, while the scatterplot in Figure 1 confirms that screeners vary greatly in the speed at which they examine items, with many taking less than four seconds to make their deductions and others taking up to fifteen seconds to correctly reject bags. Both signal detection and two-component model measures show a marked improvement with training in the competency assessment test when taken over a year apart. Previous studies using a control group have confirmed that CAT performance is not a function of exposure to the task (Koller et al., 2008), and the median date interval of 427 days would make memorisation of the task unlikely. Therefore, we can conclude that overall x-ray image interpretation competency does genuinely improve with training, increasing the accuracy and efficiency of actual screeners’ work, ultimately increasing the security of airline operations.
References Broadbent, D.E. 1964, Vigilance, British Medical Bulletin, 21:1, 17–20 Cohen, J. 1988, Statistical power analysis for the behavioral sciences, 2nd edition, (Lawrence Erlbaum, Hillsdale, New Jersey) Cohen, J. 1990, Things I have learned (so far), American Psychologist, 45:12, 1304–1312 Cohen, J. 1994, The earth is round (p < 0.05), American Psychologist, 49:12, 997– 1003 Drury, C.G. 1994, The speed-accuracy tradeoff in industry, Ergonomics, 37:4, 747–763 Drury, C.G. 1975, Inspection of sheet metal materials: model and data, Human Factors, 17, 257–265 Green, D.M., and Swets, J.A. 1966, Signal detection theory and psychophysics, New York: Wiley Grier, J.B. 1971, Nonparametric indexes for sensitivity and bias: Computing formulas Psychological Bulletin, 75, 424–42 Ghylin, K.M., Drury, C.G., and Schwaninger, A. 2006, Two-component model of security inspection: application and findings, 16th World Congress of Ergonomics, IEA 2006, Maastricht, The Netherlands, July, 10–14
388
A. Schwaninger & A.W.J. Wales
Hardmeier, D., & Schwaninger, A. 2008, Visual cognition abilities in x-ray screening. Proceedings of the 3rd International Conference on Research in Air Transportation, ICRAT 2008, Fairfax, Virginia, USA, June 1–4, 311–316. Hardmeier, D., Hofer, F., and Schwaninger, A. 2006, Increased detection performance in airport security screening using the X-Ray ORT as pre-employment assessment tool. Proceedings of the 2nd International Conference on Research in Air Transportation, ICRAT 2006, Belgrade, Serbia and Montenegro, June 24–28, 393–397 Hardmeier D, Hofer F, and Schwaninger A. 2005, The x-ray object recognition test (x-ray ort) – a reliable and valid instrument for measuring visual abilities needed in x-ray screening. IEEE ICCST Proceedings, 39, 189–192 Hofer, F. and Schwaninger, A. 2004, Reliable and valid measures of threat detection performance in X-ray screening, IEEE ICCST Proceedings, 38, 303–308 Hofer F., and Schwaninger, A. 2005, Using threat image projection data for assessing individual screener performance, WIT Transactions on the Built Environment, 82, 417–426 Koller, S.M., Hardmeier, D., Michel, S., and Schwaninger, A. 2008, Investigating training, transfer, and viewpoint effects resulting from recurrent CBT of x-ray image interpretation, Journal of Transportation Security, 1:2, 81–106 Landauer, A.A., Harris, L.J., and Pocock, D.A. 2008, Inter-subject variances as a measure of differences between groups, Applied Psychology, 31:4, 417–422 Lui, X., Gale, A., and Song, T. 2007, Detection of terrorist threats in air passenger luggage: expertise development, Proceedings of the 41st Annual IEEE International Conference, 301–306 Michel, S., Koller, S.M., Ruh, M., and Schwaninger, A. 2007, Do “image enhancement” functions really enhance x-ray image interpretation? In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Cognitive Science Society, 1301–1306 N.A. MacMillan and C.D. Creelman, 1991, Detection theory: A user’s guide. Cambridge: University Press. Pollack, I., and Norman, D.A. 1964, A non-parametric analysis of recognition experiments, Psychonomic Science, 1, 125–126. Schaller, R.R. 1997, Moore’s law: past, present and future, IEEE Spectrum, 34, 53 Schwaninger, A. 2006, Airport security human factors: from the weakest to the strongest link in airport security screening, The 4th International Aviation Security Technology Symposium Schwaninger, A., Hofer, F., and Wetter, O.E. 2007, Adaptive computer-based training increases on the job performance of x-ray screeners, Proceedings of the 41st Carnahan Conference on Security Technology, Ottawa, October, 8–11 Schwaninger, A. and Hofer, F. 2004a, Evaluation of CBT for increasing threat detection performance in X-ray screening. In: K. Morgan and M. J. Spector, The Internet Society 2004, Advances in Learning, Commerce and Security, 147–156 (Wessex: WIT Press) Schwaninger, A. 2004b, Computer based training: a powerful tool to the enhancement of human factors, Aviation Security International, 31–36
One year later: How screener performance improves in x-ray luggage search
389
Schwaninger, A. 2004a, Increasing efficiency in airport security screening. Proceedings of AVSEC World, November 3–5, Vancouver, B.C., Canada. Schwaninger, A., Bolfing, A., Halbherr, T., Helman, S., Belyavin, A., and Hay L. 2008, The impact of image based factors and training on threat detection performance in X-ray screening, Proceedings of the 3rd International Conference on Research inAirTransportation, ICRAT 2008, Fairfax, Virginia, USA, 317–324 Smith, P.L. 2000, Stochastic dynamic models of response time and accuracy: A foundational primer. Journal of Mathematical Psychology, 44:3, 408–463.
METHODS AND TOOLS
THE HEXAGON-SPINDLE MODEL FOR ERGONOMICS R. Benedyk1 & A. Woodcock2 1
2
University College London Interaction Centre, UK Design and Ergonomics Applied Research Group, Coventry University, UK
Using a systems view of a work process, a task consists of a number of transformative interactions, performed simultaneously or successively, to a given level of effectiveness, and taking place at the interface between the operator and key components of the system; these encompass not only traditional hardware, software and workstations, but social, managerial, infrastructure and peripheral factors. We propose the Hexagon – Spindle Model for Ergonomics, which represents the multiple factors influencing the effectiveness of interactions at the interface. The model is holistic, multi-dimensional, task-related and transferable across a range of settings. It extends to characterise a time base for serial and simultaneous tasks and highlights areas where operator/system conflicts may arise. We illustrate analysis tools for the application of the model in evaluation and design.
Introduction Traditional work systems approaches in ergonomics (as described originally by Singleton, 1967 and 1974, and more recently by Dowell and Long, 1998) may be applied to the satisfactory completion of any task, not just those conducted as part of ‘formal work’. Using a systems view of the process, the task consists of a number of transformative interactions, performed simultaneously or successively, to a given level of effectiveness. The tasks are accomplished in a ‘workplace’ - the immediate environment in which the user and the work system interact. Each task is defined as one or more processes undertaken by a user or operator while in a dynamic exchange of information (an ‘interaction’) with another or with the key design features of the interface. Here the wider definition of an interface is used, referring to the ‘point of communication between two or more processes, persons, or other (physical) entities’ (Santoro, 2001). This would include concrete or virtual objects such as tools, equipment, networks, furniture, and formal and informal organisational structures. Task interactions occur in a particular setting, whether in formal spaces such as offices or factories, or informal ones such as sports venues, coffee shops, cyber cafes, or the home. They are task dependent, user specific, and subject to influence from each component of the system; not only from traditional hardware, software and workstations, but also social, managerial, infrastructure and peripheral factors. The overall accomplishment of the task is achieved by the successful and effective
393
394
R. Benedyk & A. Woodcock
completion of task interactions, each of which may be influenced by wider factors beyond the immediate control of the user. The challenge for ergonomists is to find the optimal conditions of all these factors to maximise this effectiveness.
Previous ergonomic models Traditional models of ergonomics have either represented the interactive nature of the interface or the multiple sources of influential factors. The earliest traditional model, commonly known as Concentric Rings (Shackel, 1969; Galer, 1987), represented an operator-centred focus, and the influence on the operator of factors at levels moving out from there; however, there is no further elaboration in the model of these factors in the system. Girling and Birnbaum (1988) took this further with the Elaborated Concentric Rings model, in which the influential factors in the outer rings are seen to originate from management, situational or personal sources. These divisions facilitate the diagnosis of multiple sources of design problems and the directions needed for rectification. The enhanced model has since been applied in design problems, ensuring all relevant contributing factors are included (eg Dubois and Benedyk, 1991) and in evaluation problems, providing a holistic framework for scoping the evaluation (eg Naki and Benedyk, 2002). In parallel, the development of the systems approach to ergonomics has emphasised the purpose (ie the task) and the functions to support that purpose; thus Singleton’s work (1967, 1974) began the ‘closed loop’ system representation of the task interaction at the interface. This exchange of data or information between operator and interface holds true whether operator/machine or practitioner/patient or user/tool. Later authors have developed detailed structures and behaviours to further understand the interface and the nature of this interaction (eg Dowell and Long, 1998). Combining the two approaches to form a holistic representation would seem to have some advantage: as Wilson (2005, page 10) emphasises: “Humanmachine interaction does not take place in a vacuum;…ergonomic techniques are needed to predict, investigate or develop each of the possible interactions, between person-task, person-process, person-environment, person-job, personperson, person-organisation and person-outside world.” In this regard, Singleton (1967) makes passing reference to groups of external factors that could ‘interrupt the closed loop’, and Grey (1987) expands on sources within the rings, placing the task as the innermost ring. Leamon’s (1980) model comes closest to integration, in that it places the Closed Loop inside the Concentric Rings, and attempts to indicate criteria for system effectiveness; however, it elaborates no further on the rings themselves. In addition to the need for a holistic representation, there is a pressing need for an ergonomic model that can move beyond a systematic construct, to act as a useful tool for practitioners, able to demonstrate the ergonomic approach. Moreover, no models as yet embrace the complex ergonomic challenges arising from multiple users, from simultaneous or successive tasks, from peripatetic users or from changes in user characteristics over time. The Hexagon-Spindle model presented begins to address all these.
The Hexagon-Spindle model for ergonomics
395
The Hexagon Model The Hexagon Model has elaborated the Enhanced Concentric Rings model by firstly providing a clearer division of the sectors and the levels; secondly by making explicit six directions for task interactions and the factors of influence; and thirdly, by differentiating between the constant characteristics of the user (common to any task), which are in the central hexagon of the model, and the individual variables that arise only for that task, which feature in the personal sector. The model was first developed in order to elucidate the factors which effect task interactions in learning environments. Using this domain provided a series of specific tasks, interactions and multi-level factors which could be used to validate the model. These developments have been reported by Benedyk, Woodcock and Harder (2006, 2009), and Woodcock, Woolner and Benedyk (2009). In this paper, we wish to test the viability of the generic model by returning it to more traditional domains, using the example of a re-design of a cockpit simulator instructor workstation. The generic form of the Hexagon Model is shown in Figure 1. For a specific task, the user performs transformative interactions (in line with his/her requirements) with the interface at the level of the workstation; the workstation consists of all the immediate and relevant tools, materials, technology, furniture, co-workers and other people in the vicinity, facilities and resources. This wider definition of the interface is compound and complex; it is not just limited to the human-machine interface, and its interactions are subject to influence (both positive and negative) from multiple factors at the workplace, work setting and external levels. The subdivisions to the outer rings intend to ensure a sharp focus of analysis, greater rigour and consistency across applications. The model further encourages breadth of consideration, so that all possible influences are addressed. Contextual factors need to be set each time the model is applied, specific to the application under consideration. For a more detailed description of the parts of the model, the reader is referred to Benedyk et al. (2009). To perform a Hexagon Analysis, which is the first stage of the tool for practice, the model is transformed into a matrix (Figure 2) and the key factors of potential ergonomic influence noted for each cell ie, any factor likely to impact on the effectiveness of the task interaction. Every cell in the Analysis Table must be given consideration; this is one of the benefits of the technique, encouraging a holistic approach. Note that there is no attempt at quantification or comparison here; the inclusion of a factor simply gives it potential significance. If the analysis has an evaluation purpose, the factors noted may be rooted in actual observations of the task in context. However, more often the factors must be extracted from a focused creative discussion between ergonomist and user or designer. As each analysis is task dependent, the choice and relative weight of factors in the different cells will vary. The acceptable level of effectiveness also needs to be set in advance, using relevant ergonomic criteria and priorities appropriate to that work system. The ergonomic practitioner’s judgment is needed here. From the range of influencing factors identified, those most critical to task effectiveness are extracted, and an Application Format table (Figure 3) constructed, in which the potential ergonomic problems due to each factor are identified, and
396
R. Benedyk & A. Woodcock
Us Pers er Ind onal ivid Se ua ct o lF r ac tor s
s tor r ac cto F Se ure al truct on at i fr as nis t In ga en Or onm ir nv
Work-Setting Level Workplace Level Workstation Level
kE
External Factors
s Ta
Task Environment Management Factors
l cia
So
Task Task interactions interactions
s
tor
ac
pF
rou
dG
an
Pro Ma duc ter t De ials si ) F gn ( ac tor Tools s an d
User
Task Design Factors
Contextual Sector Legend for Figure 1: Shaded areas represent levels; Dotted lines represent porous boundaries; Hashed area represents task interactions at the user/workstation level interface; Sectors provide groupings for sources of factors that potentially influence the task interactions
Figure 1. A generic Hexagon Model.
solutions or interventions suggested. The emphasis of the analysis is on a holistic approach to design and evaluation which recognizes where there are mismatches between ergonomic requirements for effective interaction, and features of the actual system. These issues can then be given professional consideration, with the application of ergonomic techniques as necessary, and recommendations for improvement put forward. Figure 2 presents an exemplar of the Hexagon Analysis Table; the data here were used for the ergonomic evaluation and re-design of the physical ergonomic aspects of an instructor workstation in a cockpit motion simulator. Figure 3 presents the Application Format table, showing an illustrative analysis for the same example, and highlighting the interventions necessary to address the priority matters of ergonomic concern for this workspace design.
Developments in instructor workstations in simulators Provision of facilities for simulators; technical support Overall space, accessibility, temperature, acoustics, lighting, vibration Installation, support and power supplies affecting individual workstation; task lighting
Business funding, policy, standards for aviation training
Allocation of budget, cost/ benefits, for upgrading facility
Procurement and planning choices for design and facilities in training simulators
Procurement choice of individ'l workst'n eg adjustability, orientation, size – suit to cockpit
External Level
Task Setting Level (Pilot training centre)
Task Work place Level (Cockpit simulator)
Task Workstation Level (Instructor’s workstation)
Instructor workstation interface design functionality wrt required task performance; frequency/extent of use
Style of delivery of instruction for these tasks; equipment and sightline requirements for these tasks
Syllabus and content of instruction sessions; interaction of instructor with pilot
Best practicein simulator training facilities and in instructor facilities
Task Factors
Crowding factor; relation of instructor workstation to pilot workstation Flexibility of posture, static loading factors imposed; effect of vibration on balance, vision and well-being
Instructor workstation interface design features and their usability; sightlines between instructor and pilot’s instruments
Attitudes of instructors to different simulator workstations
Simulator instructor recruitment and training
Sightlines and engagement (both required and unwanted) between instructor and pilot
Potential interference of instruction with fidelity of cockpit simulation
Attitudes and preferences of pilots interacting with instructors
Cultural (national and business) expectations of simulator facilities
Social Factors
Personal Sector Individual Factors
Suitability of workplace configuration; effect of environmental factors on usability
Age andantiquity of simulator facilities; flexibility of use of space
Trends and standards in use of integrated cockpit instructor workstations
Tool factors
Contextual Sector
Figure 2. The Hexagon analysis table: an analysis table for a cockpit flight instructor’s tasks – this figure shows a partial analysis, giving examples filled into the table for physical ergonomics factors relevant to the ergonomic design of a flight instructor workstation.
Stable personal characteristics, eg anthropometry, skills, abilities, preferences, expectations and knowledge of particular instructor, Interface User Level (Instructor) in relation to these tasks in this cockpit simulator; individual susceptibility to effects of vibration
Interaction Level (Task interface)
Infrastructure Factors
Organisational Sector
Management Factors
For time t1
The Hexagon-Spindle model for ergonomics 397
Instructor workstation is used by many different instructors on different occasions Given the constrained space in the work area, can the full range of sizes of people who may be instructors be accommodated? Ensure anthropometry applied to this design is valid for this user population, and build in adjustability where possible.
Instructor must monitor both instructor displays and pilot displays Instruction takes place in darkness and during flight motion Can the instructor keep sightlines and focus on both displays despite obstructions and distance? Do the darkness, flight motion and isolation of this workplace cause ill effects for these operators?
Carry out user trials to test that proposed design and layout meet visual task requirements, and build in adjustability where possible. Ensure potential users test their tolerance to the environmental conditions. Ensure operators are adapted to reduce motion sickness.
Simulator must be as similar to cockpit as possible, so space for instructor is very limited Movement of flight causes vibration Are postures too constrained? Could emergency egress be compromised? Can the operator still makefine eye and finger movements without error?
Assess potential risk of postures and problems with egress, and reduce risk by operator adjustment of layout.
Motion Simulator instruction must replicate Static Simulator Instruction
Do the ergonomic requirements for the instructor in the different facilities cause a conflict?
Ensure comparison of Motion to Static simulator is not compromised by ergonomic interventions
Clients from different nations hire this training facility and require different configurations of cockpit to suit.
Different configuration simpose variations in requirements which if not met could impact optimal use of the facility bysome users
Ensure use of instructor workstation is compatible with all possible clientpreferred cockpit configurations without loss of task quality
Examples of influencing factors at time t1
Examples of ergonomics issues at time t1
Examples of ergonomics approaches to solutions
Figure 3. The application format for the Hexagon-Spindle model: an illustrative example, using the exemplar from figure 2.
Design displays and controls that do not require fine movements. Provide limb supports. Ensure rest breaks are taken outside the work environment.
Operator
Workstation
Workplace
Work-Setting
External
Levels:
398 R. Benedyk & A. Woodcock
The Hexagon-Spindle model for ergonomics
399
Legend for Figure 4: t = point in time; x = critical factors on each hexagon; Hashed area = common task interactions.
Figure 4. A Spindle Model.
The Spindle Model In the second stage of the analysis, multiple Hexagons are combined and compared, using a Spindle Model (Figure 4), in order to integrate user requirements over time. Tasks are rarely undertaken singly; the user might move from task to task consecutively, or perform more than one task concurrently. These tasks have the same user in common, and often the same external environment, perhaps even the same organisational management, but they may require different equipment, different settings and entail different user requirements, at different times. Because they are not entirely independent of each other, and relate along a time dimension, it can be useful to see their hexagons arranged as ‘plates’ on a ‘time spindle’ as in Figure 4. The commonality of the user links them at the centre. For serial tasks, the hexagons will be arranged spaced out along the time dimension as in Figure 4, and for concurrent tasks they will be ‘stacked’ at a single time point. Analysis involves a comparison of the key priority factors (the X’s in Figure 4) on each of the hexagons on the spindle, in order to identify conflict or congruence. The spindle thus can hold user task-independent needs constant while varying user requirements by task-dependent capabilities. Provision of generic facilities can sometimes compromise the specific user needs. Thus, moving from one task to another or attempting consecutive tasks in the same workstation may entail different user requirements – with the result that the workstation or the materials may be ergonomically optimal for this user for one task but not for others. Similarly, moving a peripatetic workstation (such as a laptop) from one setting or context to another can reveal conflicting requirements for optimal use, even for the same user.
400
R. Benedyk & A. Woodcock
The same analysis technique, stacking the hexagons, can compare different users doing the same task in the same workstation, and accommodate individual differences; highlighting the need for adjustability. For example, in the analysis in Figure 3, if there were two Cockpit Instructors using the same workstation at different times, their individual needs would both have to be accommodated by the workstation design. This perspective can also allow us to return to the wider focus at the workplace level. A single workplace supports not only the tasks of the system user, but also the tasks of the supervisor, the cleaning and maintenance tasks of the janitor, etc; one environment, several users. They have contrasting tasks, and thus demand contrasting user requirements from that environment. The Hexagon-Spindle model would allow us to identify points of conflict that would lead to ergonomic concern (such as increased risk) in such shared environments. Further, the model can examine the same user over time by comparing factors in the central user hexagon, and point to the changes in the interface necessary to accommodate time-dependant factors such as fatigue, aging, or pregnancy. Any of the factors in the model can be held constant, and any others varied, and the effect on the task interaction quality or the costs to the user can be examined.
Conclusion The Hexagon-Spindle model provides a generic framework that enables a practitioner to take a multi-factorial, holistic approach to the design and evaluation of different forms of task material, aids, devices and environments whilst taking into account the requirements of different types of users and tasks. Its approach will lead to a more integrated understanding of each scenario, and an increased confidence in the likely effectiveness of ergonomic interventions and design changes. This is particularly critical in complex settings with multiple tasks, wide ranges of users and budget-limited facilities, which can lack a holistic ergonomic focus. The flexibility of the spindle provides an opportunity to think outside the rigid box that can sometimes limit other models. A working day for many people now comprises sets of activities, which may or may not occur in the same place, with the same or different colleagues or using the same or different equipment. To create a truly effective working environment requires not just one, but a series of effective environments to be constructed. The Hexagon-Spindle model explicitly focuses on the effectiveness of the interactions for a single task, thus clarifying conflicts between different task needs. It evaluates the whole complex, multifactorial space as defined by the extended concentric rings and the influence of all factors, whether supportive or limiting, on the task interactions. The Spindle allows for the inclusion of an extra dimension in which to explore and resolve conflicts and priorities over time, over space, or between tasks or users. Not only may the same environment have to cater for the needs of varying users, but also for the other work undertaken in the same environment by other stakeholders. Most importantly, the model extends to provide a usable ergonomics analysis and demonstration tool.
The Hexagon-Spindle model for ergonomics
401
References Benedyk, R., Woodcock, A. and Harder, A., 2006, Towards a new model of Educational Ergonomics. In: Pro. 2nd Conf on Access and Integration in Schools, (Coventry University, UK). Benedyk, R., Woodcock, A. and Harder, A., 2009, The Hexagon-Spindle Model for Educational Ergonomics, accepted for publication in Work Journal. Dowell, J and Long, J., 1998, Conception of the cognitive engineering design problem, Ergonomics, 41.2, 126–139. Dubois, S. and Benedyk, R., 1991, An ergonomic review of the usability of the postal delivery bag. In: Designing for Everyone, I. Queinnec and F. Daniellou, Eds., (Taylor and Francis, London). Galer, I., Ed., 1987, Applied Ergonomics Handbook, (Butterworths Scientific, Guildford). Girling, B., and Birnbaum, R., 1988, An ergonomic approach to training for prevention of musculoskeletal stress at work, Physiotherapy 74.9. Grey, S., Norris, B. and Wilson, J., 1987, Ergonomics in the electronic retail environment (ICL UK Ltd, Slough, UK). Leamon, T., 1980, The organisation of industrial ergonomics – a human machine model, Applied Ergonomics, 11, 223–226. Naki, M., and Benedyk, R., 2002, Integrating new technology into nursing workstations: can ergonomics reduce risks? In: Contemporary Ergonomics 2002, P. McCabe, Ed., (Taylor and Francis, London), 57–61. Santoro, D., Snyder, R., Snyder E., 2001, FDA-Speak: A Glossary and Agency Guide, Informa Health Care, ISBN 1574911295, 9781574911299. Singleton, W., 1967, The systems prototype and his design problems, Ergonomics 10.2, 120–124. Singleton, W., 1974, Man-Machine Systems, Penguin. Shackel, B., 1969, Workstation analysis – turning cartons by hand, Applied Ergonomics 1.1, 45–51. Woodcock, A., Woolner, A. and Benedyk, R., 2009, Applying the HexagonSpindle Model for Educational Ergonomics to the design of school environments for children with autistic spectrum disorders, accepted for publication in Work Journal. Wilson, J., 2005, Methods in the understanding of human factors, Chapter 1 in Wilson, J. and Corlett, N., (eds) Evaluation of Human Work, 3rd Edition Taylor and Francis.
PRIMARY INDUSTRIES
A BIOMECHANICS WORKLOAD ASSESSMENT OF FEMALE MILKING PARLOUR OPERATIVES M. Jakob1 , F. Liebers2 & S. Behrendt2 1
Leibniz-Institute for Agricultural Engineering Potsdam-Bornim e.V., Potsdam, Germany 2 Federal Institute for Occupational Safety and Health, Berlin, Germany
Workers in modern German milking parlours – especially women – were found to be overrepresented in those suffering from musculoskeletal disorders (MSD) in comparison to the whole employed population. The objective of this study was therefore to assess the workstation geometry of female milking parlour operatives. Motion analysis in combination with EMG data recording of 14 muscles was used to evaluate different working heights and weights of milking units. Results showed that the optimal working height for attaching the cluster is having the teat ends at shoulder level of the parlour operative. Another important work load reduction can be achieved by reducing the weight of the milking cluster.
Introduction The evaluation of temporary disability data in Germany has shown a significant rate of worker absenteeism for female milking parlour operatives in comparison to the whole employed German population. Diagnosis related to disorders of the upper limb like enthesiopathies, shoulder lesions and carpal tunnel syndrome, but also disorders of the lower back (dorsalgia) and of lower limbs (knee osteoarthritis and meniscopathia) are the most prevalent (Liebers & Caffier 2006). A Swedish long term survey also reported increased discomfort among milkers (Pinzke 2003), associated with the change in systems. The Swedish studies showed that milking in the traditional tie stall system was associated with higher load peaks for the forearm and biceps muscles, but modern milking systems have a higher static load, higher values for wrist flexion as well as higher rapidness and repetitiveness (Stal et al., 1999; Stal et al., 2000). The reported discomfort and the significant number of absentee workers in Germany is indicative of the situation in modern milking parlours. The work situation in dairy farming has changed over the past decades due to mechanisation and automation. Physically demanding work tasks like lifting and carrying buckets full of milk, characteristic of traditional tie stall systems, have nearly disappeared, but at the same time the variety of tasks has decreased. The number of cows milked per hour has increased rapidly from approximately 40 to 100 cows with automation. The expectations on the worker’s performance are still increasing due to ongoing mechanisation. 405
406
M. Jakob, F. Liebers & S. Behrendt
The reduction of risk factors for back injuries, namely carrying full buckets of milk, as well as squatting next to the cow while attaching the milking cluster or milking by hand, have obviously reduced the dynamic physical workload. This is contradictory to the constant or even increasing rates of work absenteeism as a symptom of increasing discomfort. MSD cover a number of injuries and disorders of the muscles, tendons, nerves, etc. They are strongly linked to known risk factors or hazards in the workplace. Some of these hazards are found in the work situation of modern milking parlours. For example the necessity of using the same muscle groups over and over again, limited resting periods, large volumes (comparable to assembly lines), little task variation and poor workstation design (no adjustable heights, excessive reaching distances, lifting heavy tools). The aim of our study was to quantify the impact of MSD hazards and related concerns for different working heights and two weights of milking units. The weight of milking units available on the market ranges from app. 1.4 kg to up to 3.5 kg. Therefore, the force on the forearm, which has to hold the unit underneath the udder, while the cups are attached one by one, can vary extremely depending on weight of the unit and at the same time on body posture. The theoretical moments of torque based on the measured length of lever arm, which were observed while attaching a milking unit, ranged from 4.5 for a light to nearly 9 Nm for a heavy milking cluster (Jakob et al., 2007). The body posture therefore influences performance, muscular strength and endurance. The work height geometry in milking parlours is influenced by the individual body heights of the workers and the varying positions of the cow’s udders. The depth of the pit is mostly fixed and has to suit all employees ranging from about 160 to 190 cm body height. If it was possible to change the depth of the pit floor, the height may have to suit more than one worker at the same time. There is no guideline on how to adjust the proper floor height, therefore the objectives were to determine the effects of different ergonomic settings in milking parlours concerning the teat height in relation to shoulder height.
Methods The study was performed in a laboratory setting (see figure 1a) using an artificial udder to be able to control and adjust different working heights. In addition to this, platforms were used to vary the floor height. The rest of the equipment was identical to a 30◦ herringbone milking parlour; vacuum was also available to keep the teat cups attached to the artificial udder. Apart from the working heights two different milking units were tested. The light one weighed 1.4 kg and the heavy one 2.4 kg. The six settings (3 heights × 2 weights) were repeated 15 times each. Duration per repetition was fixed to one minute. During this period the worker attached the milking cluster, took it off after half a minute and spent the rest of the time waiting for the next work cycle. Attaching the cluster meant grabbing the milking unit with the left hand and holding it underneath the udder while the right hand was attaching
Biomechanics workload assessment: Female milking parlour operatives
Table 1.
Light milking unit Heavy milking unit
407
Experimental matrix.
Above shoulder level
At shoulder level
Below shoulder level
A1 A2
B1 B2
C1 C2
Figure 1. a (left) + b (right): a) Experimental setting working height above shoulder level with light milking unit showing the marker positions b) Muscle groups for surface electrodes used for electromyography measurements. cup by cup. For each person the udder height (teat ends) was adjusted above, at and below shoulder level (see table 1). The experimental matrix was randomised following a multifactorial variation of the settings. Six experienced female milkers were included in the measurements. The body height of the test persons ranged from 157 cm to 176 cm, their age from 25 to 39 years and their body mass index from 19 to 31. EMG data were recorded for 14 relevant arm and back muscles on the left and right side of the body. The heart frequency was recorded along with the EMG data. At the same time the body posture was recorded, while the worker was attaching the cups, by using video cameras and recording the positions of the eight retroreflective markers (see figure 1). Finally the test persons were asked to rate the perceived exertion following the 15 point Borg-scale. The project was approved by the Ethics Committee of the Charité-Universitätsmedizin Berlin and all subjects signed informed consent forms. The statistical analysis was carried out for all outcomes using general linear modelling (GLM, SPSS). Significance was tested for working height and weight of milking unit.
408
M. Jakob, F. Liebers & S. Behrendt
3-D Motion analysis Two Canon XM 2 cameras were used to record the work process. The applied motion analysis system was able to generate 3-D room coordinates from the marker positions. This is done with two simultaneous videos by tracking algorithms of the Simi Motion®software. The data allowed calculating different angles to describe the body postures, i.e. the upper arm elevation, the bending forward, twisting and bending sideward. The evaluation of the body postures was done according to DIN EN 1005-4, ISO 11226 and RULA (McAtamney & Corlett 1993). The markers were attached to the top of the clothes, using eight positions (see figure 1a). Three markers were fixed along the spine (C7, Th12, S1), one on the back of the head, one on the acromion, one on the trochandor major, one on the epicondylus and one on the wrist. For better contrast, black clothes were used.
Electromyography (EMG) Electromyographic measurements were carried out at the same time to be able to correlate the body posture and the muscular activity for the different ergonomic settings. The electromyographic data was collected via surface electrodes for 14 muscle groups on both sides of the body (see figure 1b). In order to rate the muscular activity, prior to testing the subjects held specified body positions designed to elicit the level of maximal muscle activity.
Borg scale The third method applied to determine the physical activity intensity was the Borg Rating of Perceived Exertion (RPE). Based on the 15 point scale, the test persons were asked three times per setting how hard they judged the work.
Results Based on the measurements the following eight assessment criteria were defined: Self evaluation of perceived exertion, duration of muscular activity, trunk inclination, upper arm elevation, side bending and torsion, heart frequency and the integrated EMG of 14 muscle groups. The biggest emphasis in this article is put on presenting the body posture.
Borg scale The mean perceived exertion (BORG rating scale) was 12 (light to somewhat hard) for all task variations. The mean for the use of a heavy milking unit was assessed with 13.07 points (somewhat hard) and significantly higher than the use of a light milking unit with 10.82 (light to very light). The range of both values described by the standard deviation was 0.95 for the heavy weight and 1.24 for the light unit. The lowest perceived exertion was assessed for working at shoulder height with a light milking unit (9.45 points, very light).
Biomechanics workload assessment: Female milking parlour operatives
409
The rate of perceived exertion was strongly affected by the weight of the milking unit and less by the working height. The heavy milking unit increased the perceived exertion by two points within the same working height. Although this is a subjective measure, the test person’s exertion rating provided a fairly good estimate for the overall evaluation.
EMG Typical muscular activities were measured on the right side of the body (preferentially shoulder and neck muscles), as well as on the left side of the body (preferentially chest and lower arm muscles). The use of a heavy milking unit increased the mean muscular activity in all muscles of both upper extremities and the back between 5 to 30% independent of the working height. Postural changes corresponded to muscular load. Working above shoulder level required the elevation of the arms and increased the muscular exertion of shoulder and neck muscles. Working below shoulder level was associated with higher activation of the lower back muscles. The lowest mean muscular activity was recorded while working at shoulder level with a light milking unit. There was an influence of the working height, but that was not necessarily statistically significant.
Body posture RULA, ISO 11226 and DIN EN 1005-4 were applied to trunk inclination, head inclination and upper arm elevation. For this, the relevant body segments were equipped with markers (see figure 1a). The planes of the reference system were also used for the determination of specific angles. Torsion and side-bending were only considered if the measured values exceeded 10◦ in either direction. A significant influence of the weight of the milking unit was given for the torsion. While side-bending showed large variation, weight and working height were not statistically significant (see figure 2). Trunk inclination and upper arm elevation were strongly influenced by the working height (see figures 3 and 4). Only the upper arm elevation of the left side was taken into consideration. The left arm held the milking unit at a constant distance to the udder, which can be considered a static working posture when taking longer than four seconds for each cycle. Each holding time was followed by a recovery time. If the teat was above shoulder level, the upper arm elevation showed the highest values (see figure 4) whereas the teat below shoulder level induced the highest values for the trunk inclination (see figure 3). The interpersonal differences were fairly large, but all participants showed the described trend.
Duration of muscular activity The average duration for attaching a milking unit was 13.7 seconds, independent of working height or weight of the unit. Attaching the heavy milking unit took between 0.5 and 1 second or rather 5% longer than the handling of a light milking
410
M. Jakob, F. Liebers & S. Behrendt
9 Side-bending Torsion
8 7
Duration in seconds
6 5 4 3 2 1 0 A1
A2
-1
B1
B2
C1
C2
Experimental setting
Figure 2. Average time in seconds and standard deviation when torsion and side-bending of the trunk exceed 10◦ while once attaching the teat cups (see table 1). 25
Angle in (°)
20 15 10 5 0 A1
A2
B1 B2 Experimental setting
C1
C2
Figure 3. Average trunk inclination and standard deviation for the different experimental settings (see table 1). unit. Working at shoulder level was performed in the shortest time, with an average of only 12.7 ± 0.6 seconds.
Blood pressure and heart rate Blood pressure and heart rate did not differ between the task variations and only slightly increased in comparison to rest conditions (low physical exertion).
Biomechanics workload assessment: Female milking parlour operatives
411
45
Angle in (°)
40 35 30 25 20 15 A1
A2
B1
B2
C1
C2
Figure 4. Average upper arm elevation and standard deviation for the different experimental settings (see table 1).
Discussion As expected, working with a heavier milking unit was associated with higher physical and perceived exertion. Subjects preferred working with a light milking unit at shoulder level. The analysis of the muscular exertion underlines this fact. On the other side, using heavy milking units and working above or below shoulder level increased muscular exertion. Ergonomic recommendations and milking machine design need to take these facts into consideration. Heavy milking units are very common in practice, because the milk is thought to be released more completely and faster through increased weight on the teats. At the same time worker comfort declines the heavier the milking clusters are. The evaluation of the body posture depends on the standards applied. The average trunk inclination for all test persons was within the acceptable range according to all the standards named (RULA, DIN and ISO). One test person exceeded the acceptable range of 20◦ trunk inclination when working below shoulder level. The upper arm elevation did not exceed 45◦ on average for all test persons, which is a limit for RULA classification. DIN ISO 11226 requires the evaluator to consider the holding time, if the upper arm elevation exceeds 20◦ without full arm support. This was the case with all experimental settings. In addition, a weight is held while lifting the arm with the moment of torque increasing with both the weight and upper arm elevation. The impact of the weight was reflected in the increase of the perceived exertion, as well as the increased muscular activity. Because of the movement of upper arm elevation and trunk inclination in the opposite direction when working height is varied, it was not possible to determine the best setting by only using these parameters. More criteria must be included. For the final evaluation the average values for the evaluation criteria were put into a score chart and the average was calculated (see fig. 5). The lowest value always scored with 1 and the highest with 6.
412
M. Jakob, F. Liebers & S. Behrendt
5
Ranking
4 3 2 1 0 A1
Figure 5.
A2
B1 B2 experimental setting
C1
C2
Final ranking of experimental setting regarding all evaluation criteria.
The high level of MSD amongst milking parlour workers will not of course be due entirely to the geometry in the milking shed design. Other changes associated to increased mechanisation, such as reduced task variety and task autonomy, longer working hours and psychosocial adjustments as milking becomes more specialised may all be factors. Work organisation elements are however beyond the scope of this study.
Conclusions The ergonomic set up in a milking parlour depends on the body height of the milker, the dimensions of the cow and the construction of the parlour. The situation changes slightly with every cow. It is therefore very hard to analyse the general risk of musculoskeletal pain or fatigue for someone milking cows. A sole body posture analysis is also not sufficient to analyse the impact of the named changing parameters. When work is done above shoulder level the arms have to be raised and the back is upright, when the udder is below shoulder level the behaviour is in the other way around. Based on the final ranking a working height on shoulder level would be placed in the middle if only body is contemplable. The whole measured range of variation in body posture was approximately 10◦ from A to C for the trunk inclination and 20◦ for the upper arm elevation. The body posture followed the expected patterns, but the absolute differences were too small to be detected by pure observation. Therefore applying a precise motion analysis system is necessary. The combination of EMG and self evaluation allows the judgement of comfort for different settings similar to human models used for cab design, etc. One of the conclusions is that the teat end should preferably be at shoulder level to guarantee the lowest perceived exertion. This is of course hard to put into practice at this point in time, because even if one had an adjustable floor,
Biomechanics workload assessment: Female milking parlour operatives
413
the adjustment could not be realised for every cow or for two workers of different body heights working at the same time. Another important finding was that the weight of the milking unit significantly influences the workload. This should be considered for new milking cluster design. There are plans to continue these investigations and add a new milking system to the experimental design, where the four teat cups are mechanically isolated and therefore there is no need to hold them with one arm underneath the udder, thus reducing static working postures. Finally, this is the first study known that evaluates the ergonomic design of milking parlours. This helps farmers in designing new milking parlours to best suit their population of cows and workers.
References DIN EN 1005-4: Sicherheit von Maschinen – Menschliche körperliche Leistung – Teil 4: Bewertung von Körperhaltung und Bewegungen bei der Arbeit an Maschinen. Deutsche Fassung EN 1005-4: 2005. ISO 11226:2000: Ergonomics – Evaluation of static working postures, draft. Jakob M, Rose S, Brunsch R: Einfluss der Melkstandausstattung auf die Arbeitsbelastung des Melkers. Z. Arb. Wiss., 61, 173–181, 03-2007. Liebers F, Caffier G: Muskel-Skelett-Erkrankungen in Land- und Forstwirtschaft sowie Gartenbau – Diagnose- und berufsspezifische Auswertung von Arbeitsunfähigkeits-daten. Arbeitsmed. Sozialmed. Umweltmed. 41, 3, 2006: 129. McAtamney L.; Corlett EN: A survey method for investigation of work-related upper limb disorders. Applied Ergonomics 1993, 24(2), 91–99. Pinzke S: Changes in working conditions and health among dairy farmers in southern Sweden. A 14-year follow-up. Ann Agric Environ Med, 10, 185–195, 2003. Stal M, Hansson G-A, Moritz U: Wrist positions and movements as possible risk factors during machine milking. Applied Ergonomics 1999, 30, 527–533. Stal M, Hansson G-A, Moritz U: Upper extremity muscular load during machine milking. International Journal of Industrial Ergonomics 2000, 26, 9–17.
SWEDISH PERSPECTIVES ON ERGONOMICS IN THE PRIMARY SECTOR P. Lundqvist Department of Work Science, Business Economics and Environmental Psychology, Swedish University of Agricultural Sciences, Alnarp, Sweden The primary sector, defined as agriculture, horticulture, forestry, fishing and related industries has been in focus of ergonomic research in Sweden for a number of years. The peak period of research was in the 1980ies and 90ies. Since then the major efforts have been done in agriculture and in the forestry sector. Research on ergonomics in agriculture is active in Sweden, with a focus on ergonomics and animal production. Several studies over the years have involved epidemiological, clinical and ergonomic studies in different types of milking parlours, technical and housing solutions. In recent years there has been taken a number of ergonomic initiatives for machine operators by the Swedish researchers in collaboration with the logging industry. A major EU project was the “Ergoefficient mechanised logging operations, ErgoWood” involving Sweden and other European countries. Less research has been done in horticulture and fishing although there is obvious needs for research and development. An increased level of European collaboration in the area of ergonomic research is essential in order to maintain competence in order to evaluate and develop the working conditions in the primary sector.
Introduction Complaints affecting the back, neck, shoulders, knee and other joints are examples of the most common causes of illness among the employed population. That was the major reason why Sweden in 1983, as one of the first coupntries in the world, issued a standard concerning ergonomics for the prevention of musculoskeletal disorders (Swedish Work Environment Authority, 1998). This paper has a focus at perspectives on research applied to ergonomic conditions concerning musculoskeletal disorders at work and how work environment conditions can be designed and arranged in such a way that risks of physical loads which are dangerous to health can be prevented. The primary sector, defined as agriculture, horticulture, forestry, fishing and related industries has been in focus of ergonomic research in Sweden for a number of years. The peak period of research was in the 1980ies and 90ies. Since then the major efforts have been done in agriculture and in the forestry sector. Less 414
Swedish perspectives on ergonomics in the primary sector
415
research has been done in horticulture and fishing although there is obvious needs for research and development. During the last decade major interest has been focused on accidents and injuries in the primary sector and at the present there is a national program starting up with a focus on injury prevention in agriculture using funding from EU rural development programs (Lundqvist & Alwall, 2008). There has also been major cut downs in the area of work environment in Sweden in recent years. The Swedish National Institute for Working Life had to close down 2007 and the Swedish Work Environment Authority has major reductions in budget, due to Government decisions, which has a negative impact on research and other related activities in Sweden.
Agriculture Work in agriculture has been considered to be a highly demanding occupation with work tasks that can cause musculoskeletal disorders and work disability (Lundqvist, 2000). Swedish research in this area has mainly been focusing on working with animals, but also to some extent the man-machine system in tractors.
Animal production Research on ergonomics in agriculture is active in Sweden, with a focus on ergonomics and animal production. A recent PhD dissertation presented studies on the employee perspectives on ergonomics in milk- and pig- production (Kolstrup, 2008, Kolstrup et al., 2006, 2008). Among her findings it was concluded that the livestock workers assessed their psychosocial work environment and mental health as good, although the quality of leadership, feedback and social support was experienced as being slightly poorer on dairy farms compared to pig farms. No psychosocial risk factors were identified for musculoskeletal disorders (MSD). Dairy farm workers working with healthy cows had poorer physical and mental health than those working with less healthy dairy cows. The livestock workers were contented with their psychosocial work environment; however, they reported high frequencies of MSD. The prevalence of MSD seemed to be associated with the physical rather than the psychosocial work environment. The above mentioned thesis and a number of studies are performed by a research group at the Agricultural University in Alnarp specialising in ergonomics in agriculture, with a special focus on animal production. Several studies over the years have involved epidemiological, clinical and ergonomic studies in different types of milking parlours, technical and housing solutions (Lundqvist 1988, Lundqvist et al., 1997, 2003, Pinzke 2003, Pinzke et al., 2003, Stål et al., 1997, 2003, 2004). A number of the studies have been adopted by the industry which means better products for the end users. There are also a number of ongoing studies in Sweden involving ergonomics during work with horses, one is focusing on riding instructors (Löfqvist, 2008), and another one on hand held tools for manual work operations. Ergonomic studies on work with pigs has been quite rare until recent year, but now a number of studies has
416
P. Lundqvist
shown the frequency of musculoskeletal disorders together with in-depth studies of different manual work operations (Gustafsson & Lundqvist 2003, Stål & Englund 2005, Kolstrup 2008). Another Swedish initiative have been the development of guidelines for children and adolescents on farms ( Alwall Svennefelt, & Lundqvist, 2006) giving the family directions on what work operations could be suitable for children to be involved in and what considerations needs to be taken, such as age, strength, body postures and other issues of importance.
Tractor ergonomics During recent decades the focus around machine ergonomics in agriculture has been directed towards sitting postures in tractors. Anna Torén and Kurt Öberg at the Swedish University of Agricultural Sciences in Uppsala made a number of in-depth studies (Torén 2001, Torén & Öberg 1999, Torén et al., 2002). The studies showed that ploughing was a static task and that 70% of the drivers sat with the trunk heavily twisted (hip-shoulder angles less than −30 and more than 30 degrees). Harrowing contained more variation than ploughing, but the exposure to heavily twisted trunk posture was 20%. With the use of freely swivelling saddle chairs the exposure to heavily twisted trunk postures could be reduced and even eliminated, especially during ploughing. Since these studies were performed, has this research group been split up and no more research is performed within this area.
Horticulture In the 1980ies there was performed a number of Swedish studies on the climatic effects on the greenhouse workers with physiological measurements in a climate chamber and in greenhouses (Lundqvist 1988 Lundqvist & Gustafsson 1988, Gustafsson et al., 1989). These studies showed that the level of physiological strain as measured as deep body temperature, skin temperature, sweat production and pulse rate in greenhouse work increases with rising heat stress and length of exposure. Females tended to have slightly less heat tolerance than male workers. The prevalence of musculoskeletal disorders in horticultural workers was established in a mail- in survey by Lundqvist and Pinzke (1996). The results showed that a large proportion of both male and female workers had problems in the musculoskeletal system. Compared to reference data from other occupations, pain and discomfort were especially frequent in the neck, shoulders, lower back and knees. In addition females reported severe problems with the wrists/hands. The general conclusion of the study was that a reduction of musculoskeletal problems in horticulture is important but requires a series of preventive measures, such as improvements of the physical work environment including material handling equipment, development of work organisation focusing on forms of remuneration and the scheduling of work tasks and hours. Finally it was pointed out the need for training of personnel in work methods and techniques. Horticulture involves a lot of manual work operations and is less mechanised compared to other parts of the primary sector. Greenhouse work involves a lot of
Swedish perspectives on ergonomics in the primary sector
417
strain in awkward postures and this goes for a lot of field work as well, working at a ground level. These issues were discussed during an international seminar in California. Two presentations described the problems from a Swedish perspective (Lundqvist 2004a, b), this is obviously an area which needs international collaboration in order to find effective solutions, maybe within a joint EU-program.
Forestry In recent years there has been taken a number of ergonomic initiatives for machine operators by the Swedish researchers in collaboration with the logging industry (Eklund et al., 1994, Attebrant et al., 1997, Synwoldt & Gellerstedt 2003). A major EU project was the “Ergoefficient mechanised logging operations” (ErgoWood, 2008), involving Sweden and other European countries, funded by the European Commission to develop guidelines on ergonomic matters for European users, buyers and manufacturers of forest machines. This work promoted the development of safe and efficient forest machines, which are easy to use and maintain, as well as improving the sustainability in human resources. The project also involved developing and sharing good examples of work-crew building, work-shift scheduling, job rotation and work enlargement in logging operation. Different ways of organising forest work was also investigated and assessed. A large amount of publications was published (ErgoWood 2008, Bohlin & Hultåker 2006, Gellerstedt et al., 2006, Hanse & Winkel 2008), including reports in different languages and a handbook which was developed focusing on work environment, management, tools included costing of illness and preventive measures for forestry small and medium sized enterprises (SME). Besides focus on machine operators there have also been studies on heat stress in forestry work (Wästerlund et al., 2004). This is not often a major problem in Sweden, but in many countries with tropical climate is this essential.
Fishing industry Working conditions in the fishing industry is often severe. Working hours are long and irregular, sleep deprivation is a problem. The work is heavy and noise as well as climate conditions are major problems. Research on ergonomics and working conditions in the Swedish fishing industry has mainly been performed by Marianne Törner and colleagues at University of Gothenburg. Major findings were presented in her dissertation (Törner, 1991) were she concludes that subjective musculoskeletal symptoms were common, mainly from the low back, neck/shoulders and knees. The load on the articular system was increased in conditions of ship motions. Even in mild weather conditions, lumbar compressions of a magnitude large enough to create a risk for back injuries were present. Factors connected to the high load constituted the third most common reason for staff turn-over in fishery, preceded by economics and social reasons. Musculo-skeletal and other medical reasons were more often reasons for leaving fishery among men 40–50 years old, than among the younger men.
418
P. Lundqvist
During recent years ergonomics improvements have been introduced in the fishing industry, but much remains to be done and can be done as shown in studies by Törner et al. (1988, 1994) concerning mechanization of catch and gear handling, work organisation and hull designs aimed at reducing ship motions. Research on ergonomics in the fishing industry has not been a prioritised area in Sweden during the last decade.
Discussion and conclusion This limited review of ergonomic research within the primary sector in Sweden shows clearly that it is difficult to maintain and develop competence. It has been shown that ergonomic research starts up with a focus on a certain area of the primary sector, but it is not very often that we can see long lasting strength. In this paper it has been shown that the forestry sector has maintained strong and active research agenda. The joint European project “ErgoWood” seems to be a successful model for maintaining national research in an international context. The other successful example is the research group on ergonomics in animal production. They have maintained a focus over the years and have published in international journals, which is essential. The next step is taking part in a EU project called “Good practices in agriculture: social partners participation in the prevention of musculoskeletal disorders” in collaboration with colleagues from Poland, England, Spain and Belgium. The conclusion has to be: maintain and develop a research area with in-depth studies and take part in joint international networks and projects. There is also much structural changes going on and it is important that ergonomic considerations are taken in order to create an environment which is attractive as a future work place, both for the younger and the aging generation.
References Alwall Svennefelt, C & Lundqvist, P. 2006. Swedish Guidelines for Children’s Agricultural Tasks – Part II. In: Proceedings of the 2006 National Institute for Farm Safety’s (NIFS) Annual Conference, “Meeting Challenges Together,” June 25–30, Sheboygan, Wisconsin. Attebrant M, Winkel J, Mathiassen SE, Kjellberg A. 1997. Shoulder-arm muscle load and performance during control operation in forestry machines. Effects of changing to a new arm rest, lever and boom control system. Appl Ergon. 28(2):85–97. Bohlin, F & Hultåker, O. 2006. Controlling the costs of work related illness in forestry – What can the contractor do? Forestry Studies. 45: 37–48. ErgoWood, 2008. ErgoWood. Ergoefficient mechanised logging operations. Aviable from: http://www2.spm.slu.se/ergowood/index.htm (Accessed 2008-12-05). Eklund J, Odenrick P, Zettergren S, Johansson H. 1994. Head posture measurements among work vehicle drivers and implications for work and workplace design. Ergonomics 37(4):623–39.
Swedish perspectives on ergonomics in the primary sector
419
Gellerstedt, S, Eriksson, G, Frisk, S, Hultåker, O, Synwoldt U, Tobisch R, Weise, G. 2006. European ergonomic and safety guidelines for forest machines 2006. ErgoWood. Swedish University of Agricultural Sciences. Uppsala. Gustafsson, B & Lundqvist, P. 2003. Work Environment Issues in Swine Production. XXX CIOSTA-CIGR V Congress “Management and Technology Applications to Empower Agriculture and Agro-Food Systems”, Turin, Italy. Sept. 22–24 2003. Proceedings pp 941–953. University of Turin, Italy. Hanse, J.J. & Winkel, J. 2008. Work organisation constructs and ergonomic outcomes among European forest machine operators. Ergonomics 51 (7), pp. 968–981. Kolstrup, C. 2008 Work environment and health among Swedish livestock workers. Doctoral diss. Dept. of Work Science, Business Economics & Environmental Psychology, SLU. Acta Universitatis agriculturae Sueciae vol. 2008:43. Kolstrup C, Stål M, Pinzke S, Lundqvist P. 2006. Ache, pain, and discomfort: the reward for working with many cows and sows? J Agromedicine. 11(2):45–55. Kolstrup C, Lundqvist P, Pinzke S. 2008. Psychosocial work environment among employed Swedish dairy and pig farmworkers. J Agromedicine. 13(1):23–36. Lundqvist, P. 1988. Working environment in farm buildings. Results of studies in livestock buildings and greenhouses. Doctoral Dissertation. Swedish University of Agricultural Sciences. Department of Farm Buildings. Report 58. Lund. Lundqvist, P. 1999. Humans – Health, Psychological and Sociological Factors. In: The Greenhouse Ecosystem.Ecosystems of the World. Editor: Enoch, H. Z. Chapter 11. pp 251–263. Elsevier. Amsterdam. Lundqvist, P. 2000. Occupational health and safety of workers in agriculture and horticulture. Journal of New Horizons. Vol 10, No 4:351–365. Lundqvist, P. 2004a. The Scope of the Problem of Stooped and Squatting Postures in theWorkplace. International Perspectives (Sweden). Proceedings. Conference on Stooped Postures in the Workplace. Oakland, California, U.S.A., 2004-07-29–30, University of California, Davis. Lundqvist, P. 2004b. Agricultural Interventions in Sweden. Conference on Stooped Postures in the Workplace. Oakland, California, U.S.A., 2004-07-29–30, University of California, Davis. Lundqvist, P. & Alwall. C. 2008. Strategies for Injury Prevention in Swedish Agriculture. In: Proceedings of 6th International Symposium. Public Health & theAgricultural Rural Ecosystem. October 19–23, 2008. Saskatoon, SK, Canada. Lundqvist, P. & Gustafsson, B. 1988. Working environment in greenhouse – a review of Swedish research. HortScience, Vol 23(3), June 1988, sid 446–448. Lundqvist, P & Pinzke, S. 1996. Musculoskeletal Disorders in Horticultural Workers. In: A. Mital., H. Kreuger, S. Kumar, M. Menozzi and J.E. Fernandez (Editors). Advances in Occupational Ergonomics and Safety I (2 Vol.) pp. 433–438. Cincinatti. Lundqvist, P., Stål and Pinzke, S. 1997. Ergonomics of Cow Milking in Sweden. J Agromedicine. 3(2): 169–176. Lundqvist, P., Stål M. and Pinzke, S. 2003. Working Conditions & Ergonomics When Milking Cows. In: Fifth International Dairy Housing Conference. Proceedings of the 29–31 January 2003 Conference, Fort Worth, Texas, pp 59–65.
420
P. Lundqvist
The Society for engineering in agricultural, food and biological systems (ASAE), St Joseph, USA. Löfqvist C. 2008. The human-horse work environment: A Swedish perspective. Nordic Meeting on Agricultural Occupational Health, Århus, Danmark, 1–3 September 2008. Pinzke S. 2003. Changes in working conditions and health among dairy farmers in southern Sweden. A 14-year follow-up. Ann Agric Environ Med. 10(2):185–9. Pinzke S, Stal M, Hansson GA. 2001. Physical workload on upper extremities in various operations during machine milking. Ann Agric Environ Med. 8(1): 63–70. Stål M, Moritz U, Johnsson B, Pinzke S. 1997. The Natural Course of Musculoskeletal Symptoms and Clinical Findings in Upper Extremities of Female Milkers. Int J Occup Environ Health. 3(3):190–197. Stål M, Pinzke S, Hansson GA, Kolstrup C. 2003. Highly repetitive work operations in a modern milking system. A case study of wrist positions and movements in a rotary system. Ann Agric Environ Med. 10(1):67–72. Stål M, Hagert CG, Englund JE. 2004. Pronator syndrome: a retrospective study of median nerve entrapment at the elbow in female machine milkers. J Agric Saf Health.10(4):247–56. Stål M, Englund JE. 2005. Gender difference in prevalence of upper extremity musculoskeletal symptoms among Swedish pig farmers. J Agric Saf Health. 11(1):7–17. Swedish Work Environment Authority, 1998. Ergonomics for the Prevention of Musculo-skeletal Disorders. Provisions AFS 1998:1. Solna. Synwoldt U, Gellerstedt S. 2003. Ergonomic initiatives for machine operators by the Swedish logging industry. Appl Ergon. 34(2):149–56. Torén A. 2001. Muscle activity and range of motion during active trunk rotation in a sitting posture. Appl Ergon. 32(6):583–91. Torén A, Oberg K. 1999. Maximum isometric trunk muscle strength and activity at trunk axial rotation during sitting. Appl Ergon. 30(6):515–25. Torén A, Oberg K, Lembke B, Enlund K, Rask-Andersen A. 2002. Tractor-driving hours and their relation to self-reported low-back and hip symptoms. Appl Ergon. 33(2):139–46. Törner M, Blide G, Eriksson H, Kadefors R, Karlsson R, Petersen I. 1988. Workload and ergonomic measures in Swedish professional fishing. Appl Ergon. 19(3):202–12. Törner, M. 1991. Musculo-skeletal stress in fishery. Causes, effects and preventive measures. Dissertation. Department of Orthopaedics, University of Gothenburg, Sahlgren Hospital. Gothenburg. Törner M, Almström C, Karlsson R, Kadefors R, 1994. Working on a moving surface–a biomechanical analysis of musculo-skeletal load due to ship motions in combination with work. Ergonomics 37(2):345–62. Wästerlund DS, Chaseling J, Burström L. 2004. The effect of fluid consumption on the forest workers’ performance strategy. Appl Ergon. 35(1):29–36.
ROAD ERGONOMICS
A 50-DRIVER NATURALISTIC BRAKING STUDY: OVERVIEW AND FIRST RESULTS N. Gkikas, J.H. Richardson & J.R. Hill Ergonomics & Safety Research Institute, Loughborough University, UK Considering the importance of vehicle brake systems, it is surprising how little is known about the way that people operate them. Previous ergonomic studies have attempted to define the maximum acceptable resistance to depression in the pedal (Diffrient, Tilley, & Harman, 1993; Eaton & Dittmeier, 1970). Accordingly, they focussed on the responses of weak (5 percentile muscle strength) female drivers and little is known about the full range of braking response. A reexamination of this basic control mechanism is necessitated by the evolution of vehicle systems. The present paper offers an overview of a study measuring driver “pedipulation” in a naturalistic environment. Fifty-eight fully-licensed drivers drove a car for a day. The types of trip analysed included commuting to work, shopping, and picking up children from school. Measures taken included throttle pedal angle, brake pedal pressure, and clutch pedal pressure. The foot well was constantly video recorded during each trip. Main results are presented and comparisons with earlier studies are discussed.
Introduction The safe driving of any road vehicle is achieved through successful control of its longitudinal and lateral dynamics. In practice, longitudinal control is speed control and this is carried out via operation of the pedals (throttle, brake and clutch) and the gear stick (where manual transmission applies). As movement and speed define whether two vehicles may collide and the size of the impact, brake operation is of unique importance in safe driving. According to accident data, braking is the most common road user reaction (when there is any) during the course of an accident .(Gkikas, Hill, & Richardson, 2008).
Previous studies Considering the above, it is surprising how quickly research moved from studies about the actual operation of the brake pedals (Eaton & Dittmeier, 1970) to studies of braking in relation to car-following/time to collision (TTC) and Adaptive Cruise Control (ACC) specification (Abe & Richardson, 2005; Chung & Yi, 2007; Dingus et al., 1997; Lin, Hwang, Su, & Chen, 2008; McCall & Trivedi, 2007; Van Der Horst, 1990; Van Winsum & Heino, 1996; Van Winsum, 1998) In parallel, driver 423
424
N. Gkikas, J.H. Richardson & J.R. Hill
braking has been used as a measurement of distraction in related studies (Harbluk, Noy, Trbovich, & Eizenman, 2007; Morita, Mashiko, & Okada; Sato & Akamatsu, 2007). The increased popularity and acceptance of multi-level models for the driving task (Lee, 2006; Michon, 1993; Summala, 1996) and the aggregated complexity induced by the introduction of new technologies in the car could partially explain this trend, however not completely. In contrast it could be argued that nearly 40 odd years after Eaton’s study on braking capabilities of drivers, it is time for an update on a task that could have changed a lot or not at all. The present paper presents results from an exploratory study of “naturalistic” driver braking.
Method In order to explore ‘normal’ braking behaviour, 58 drivers drove an instrumented vehicle while completing various daytime trips. Trips included (but were not limited to) driving between home and work, driving children to and from school, shopping related trips etc.
Sample 58 fully licensed drivers with less than 6 points on their license took part in the study. Most of them were associated with Loughborough University or other local businesses. They ranged between 23 and 84 years in age (Mean: 35.76), between 1 and 48 years of driving experience (Mean: 16.37), between 1000 and 30,000 yearly mileage (Mean: 9733.33), between 152 cm and 193 cm in height (Mean: 171.48 cm), and between 50 and 115 kilograms in weight (Mean: 73.91). 30 were male and 28 were female. Figures 1 and 2 show the distribution in height and weight of the participants (stacked male-female groups).
Apparatus: The instrumented vehicle For the purpose of the study, a T registration plate Ford Fiesta was instrumented with sensors on the pedals and a camera in the foot-well. Tekscan Flexiforce® sensors were installed on the surface of the brake pedal and the clutch. A potentiometer was installed at the centre of rotation of the throttle pedal. Flexiforce® sensors were calibrated and conditioned according to Tekscan guidelines. U12 Labjack® Data Acquisition card in conjunction with a Toshiba Tecra 3 laptop using the Data Acquisition Factory Express® software was used for data logging. In addition, a Microsoft® Litecam VX-1000 webcam was installed at the side of the foot-well, capturing video at 60 Hz.
Realistic journeys? Most typically the car was delivered to a participant’s address early in the morning to commute to work and was collected in the evening from the same place. There
A 50-driver naturalistic braking study: Overview and first results
425
15 gender female male
Frequency
10
5
0 60.00
Figure 1.
80.00 Weight in kg
100.00
120.00
Stacked histogram of male/female participants’ weight.
were, however, cases where the participant opted to collect the car from its base (Loughborough University) and bring it back in the evening as well. For practical reasons, especially in the case of the car being delivered, the delivery driver would often sit in the passenger seat. The participants were aware that the aim was to monitor their “normal” driving, but didn’t know exactly what aspect was being monitored. In summary, there is no apparent reason why the braking data collected should not be representative of the drivers’ typical braking activity whilst making the trips that were undertaken.
Measurements Both quantitative and qualitative pedal operation data were collected. The main measurements included throttle pedal angle change – mainly during release (“throttle-off ”), force (and pressure) applied on the brake pedal surface and force (and pressure) applied on the clutch pedal surface. “Throttle-off ” is the release of the throttle pedal that results in some deceleration of the vehicle, and is therefore either part of the braking sequence or a form of braking in itself. A camera in the foot well provided qualitative data on foot movement and pedal operation.
426
N. Gkikas, J.H. Richardson & J.R. Hill 10
gender female male
Frequency
8
6
4
2
0 150.00
Figure 2.
160.00
170.00 180.00 Height in cm
190.00
200.00
Stacked histogram of male/female participants’ height.
Results In observational studies of this size and this depth the amount of data captured is such that analysis can demand even more time than data acquisition itself. Only some first results are presented here. These are: descriptive exploration of brake and throttle pedal operation in relation to vehicle deceleration, comparison between genders and examination of interaction of the main variables with demographics (age, experience, annual mileage, height and weight). Figure 3 presents the distribution of average forces applied on each sensor on the brake pedal during driving (values in Newtons). To make those values more understandable and usable, we need to convert them to pressure on the pedal surface (in Pascals). Those values can be found in table 1. The average pressure on the brake pedal was 78630.18 Pa with a standard deviation of 65623.6 Pa. Group statistics indicate that female drivers tended to apply higher forces on the brake pedal with an average of 100920.85 Pascals compared to 59215.82 Pascals for male drivers (Table 2). Further analysis using aT-test indicated that the difference was statistically significant (p = 0.01). In terms of throttle-off angle change, values were strongly concentrated around the mean point. The average angle change (50 Hz sampling) was 0.51 degrees, with a standard deviation of 0.07 (table 1). The average for male drivers was 0.5 degrees and the standard deviation was 0.06 degrees while for female drivers the respective values were 0.52 and 0.08 degrees.
A 50-driver naturalistic braking study: Overview and first results
427
25 gender female male
Frequency
20
15
10
5
0 0.00
10.00 20.00 30.00 mean_pressure_between_sensors_public_road
40.00
Figure 3.
Stacked histogram of average forces applied on each sensor on the brake pedal (all values in Newtons).
Table 1.
Descriptives of pressure on the pedal and throttle-off angle change.
Mean pressure on brake pedal (Pascal) Throttle-off angle change (50 Hz) Valid N
Table 2.
N
Minimum
Maximum
Mean
Std. Deviation
58
.00
448549.68
78630.18
65623.60
56 56
.18
.56
.51
.07
group descriptives of pressure on brake pedal and throttle-off angle change for male and female drivers.
Mean pressure on brake pedal (Pascal) Throttle-off angle change (50 Hz)
Gender
N
Mean
Std. Deviation
Std. Error Mean
Male Female Male Female
31 27 29 27
59215.72 100920.85 .50 .52
41144.92 80760.97 .065 .075
7389.85 15542.46 .012 .014
Further analysis of the interaction of mean brake pedal pressure and throttleoff rate against age, experience, height, weight and annual mileage revealed no statistically significant correlations. All probability values were above the typical a = 0.01 criterion. Statistically significant correlations were found between age
428
Table 3.
N. Gkikas, J.H. Richardson & J.R. Hill
Male and female percentiles of brake pedal pressure (all values in Pascals). Gender
Percentiles
Male
Female
5 10 25 50 75 90 95
8407.64 15333.37 37503.18 48848.40 76777.07 113083.82 156585.98
33824.45 37852.10 54203.82 82101.91 120926.11 171286.11 342657.45
and driving experience (R = 0.95, p = 0.0001) and between height and weight of participants (R = 0.65, p = 0.0001). Table 3 presents the estimated percentiles of brake pedal pressure. It is obvious that female drivers were applying much higher forces on the pedal during braking. For example the estimated 5th percentile female (about 33 kPa) is more than four times harder on the pedal than the 5th percentile male (about 8 kPa).
Discussion The study presented here featured some advantages over the previously quoted studies, especially in the methods used. Participants were recruited through advertising and word of mouth, without the use of any incentives that would appeal to a specific population. A normal distribution curve fitted well on height and weight data of participants (figures 1–2). This is particularly important as size of drivers could affect their posture (Porter & Gyi, 1998) in the vehicle and thus their operation of the pedal controls. The sample size was over 50 drivers and almost every age group was represented. The first interesting result of the analysis is the difference between genders on force exerted during brake pedal operation. Considering both the results from Eaton’s experiments (1970) as well as the data provided by Humanscale pedal controls .(Diffrient, Tilley, & Harman, 1993), it is somewhat contradictory that the women exerted greater pedal force than the men. In these early studies small female drivers were used as a criterion for the minimum acceptable resistance of pedal control. Current results suggest that in everyday driving female drivers brake harder than male drivers. There seems to be a difference between what is theoretically expected according to muscle strength and what happens in action. However, it should be acknowledged that the studies measured variables related but not identical. Maximum possible force during operation, which was the scope of Eaton’s study using small females in the ‘60s, is not the same as the actual force drivers apply when they drive on a public road, in a real car, in 2008. On the other hand, if the pressure values above are adjusted to the area of the brake pedal (about 20 cm2 ), then the force values become very similar to the recommended values in
A 50-driver naturalistic braking study: Overview and first results
429
Humanscale (4–30 N). And although Humanscale’s recommendation is somewhat too wide (it actually corresponds to 1st – 10th %ile male braking force in the current study), in practical terms it is a realistic guideline as it falls within the lower range of forces encountered in the present study. The second major comment concerns the implications of these data in relation to modern active safety systems and Emergency Brake Assist (EBA) in particular. There are some field studies suggesting significant contribution of such systems in rear-end collision mitigation (Breuer, Faulhaber, Frank, & Gleissner, 2007) and extended studies have attempted to identify the distinction between “normal” and “emergency” brake characteristics for the purpose of devising successful system specification (Kassaagi, 2001; Perron, Kassaagi, & Brissart, 2001). The size of the standard deviation in force/pressure applied on the brake pedal points towards a variation that cannot be accommodated by a constant threshold of activation, as most current systems do. A quick view of table 3 indicates that the average brake pressure quadruples between 25th percentile male and 75th percentile female drivers. Thus, it is highly unlikely that a single triggering value in brake force/pressure could accommodate this amount of variance. On the contrary, the distribution of the throttle-off measurements showed strong concentration around the mean. This trend could be incorporated in the specification of relevant systems (and it already is in some versions of EBA). However, it remains to be seen what the respective values in a certain emergency braking event might be. If there is a difference between the two conditions, then due to the limited variation, throttle-off is a more promising triggering variable. In addition, as it takes place before brake application, it has a temporal competitive advantage. Overall, results suggest that throttle-off is an important component of the braking task and safe longitudinal control of the vehicle. The main limitation of this study is the use of a single vehicle by all participants. This study design allows room for possible confounding effects attributable to the particular vehicle concerned. Such a possibility could be evaluated by the use of alternative vehicles and subsequent comparison of results. Results from the early study by Eaton (1970) indicate that there is a minor increase in the forces applied on the brake pedal as the size of the vehicle increases; therefore it is desirable to put that to the test again. Ideally, the instrumentation of the actual vehicles owned and driven by the drivers should be considered in future studies. The second limitation of the study has to do with the size of the sample. Fiftyeight drivers is considered to be a potentially representative number: however a greater number and the inclusion of drivers from other areas of the country, and, ideally, of the world would be desirable. To improve confidence in the data reported so far, the sample size should be increased and other car models used in a more significantly resourced study.
References Abe, G., & Richardson, J. (2005). The influence of alarm timing on braking response and driver trust in low speed driving. Safety Science, 43(9), 639–654.
430
N. Gkikas, J.H. Richardson & J.R. Hill
Breuer, J. J., Faulhaber, A., Frank, P., & Gleissner, S. (2007). Real world benefits of brake assistance systems. Proceedings of the 20th Enhanced Safety in Vehicles Conference, Lyon. (Paper Number 07-0103) Chung, T., & Yi, K. (2007). An investigation into differential braking strategies on a banked road for vehicle stability control. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 221(4), 443–455. Diffrient, N., Tilley, A. R., & Harman, D. (1993). Humanscale 4/5/6: A portfolio of information, 4 human strength and safety, 5 controls and displays, 6 designing for people. Cambridge, Mass.: MIT Press. Dingus, T. A., McGehee, D. V., Manakkal, N., Jahns, S. K., Carney, C., & Hankey, J. M. (1997). Human factors field evaluation of automotive headway maintenance/collision warning devices. Human Factors, 39(2), 216–229. Eaton, D. A., & Dittmeier, H. J. (1970). Braking and steering effort capabilities of drivers. SAE Technical Papers, (Paper No. 700363) Gkikas, N., Hill, J., & Richardson, J. (2008). Getting back to basics: Using road accident investigation to identify the desirable functionality of longitudinal control systems. In D. de Waard, F. O. Flemisch, B. Lorenz, H. Oberheid & K. A. Brookhuis (Eds.), Human factors for assistance and automation. (pp. 203–216). Maastricht, the Netherlands: Shaker Publishing. Harbluk, J. L., Noy, Y. I., Trbovich, P. L., & Eizenman, M. (2007/3). An on-road assessment of cognitive distraction: Impacts on drivers’ visual behavior and braking performance. Accident Analysis & Prevention, 39(2), 372–379. Kassaagi, M. (2001). Caracterisation experimentale du comportement des conducteurs en situation d’urgence pour la specification de systemes de securite active. Unpublished These de doctorat, Ecole Centrale Paris, Lee, J. D. (2006). Driving safety. In R. S. Nickerson (Ed.), Reviews of human factors and ergonomics (pp. 173–218). Santa Monica, USA: Human Factors and Ergonomics Society. Lin, T. W., Hwang, S. L., Su, J. M., & Chen, W. H. (2008). Effects of in-vehicle tasks and time-gap selection while reclaiming control from adaptive cruise control (ACC) with bus simulator. Accident Analysis and Prevention, 40(3), 1164–1170. McCall, J. C., & Trivedi, M. M. (2007). Driver behavior and situation aware brake assistance for intelligent vehicles. Proceedings of the IEEE, 95(2), 374–387. Michon, J. A. (1993). Generic intelligent driver support. London: Taylor & Francis. Morita, K., Mashiko, J., & Okada, T. Theorectical analysis of delay in braking operation when drivers looking away from the road ahead. Human factors in 2000: Driving, lighting, seating comfort, and harmony in vehicle systems. (SP-1539) (pp. 57–66) Society for Automotive Engineers. Perron, T., Kassaagi, M., & Brissart, G. (2001). Active safety experiments with common drivers for the specification of active safety systems. Paper presented at the 17th International Technical Conference on Enhanced Safety of Vehicles, Amsterdam. (Paper Number 427) Retrieved from http://wwwnrd.nhtsa.dot.gov/database/nrd-01/esv/asp/esvpdf.asp Porter, J. M., & Gyi, D. E. (1998). Exploring the optimum posture for driver comfort. International Journal of Vehicle Design, 19(3), 255–266.
A 50-driver naturalistic braking study: Overview and first results
431
Sato, T., & Akamatsu, M. (2007). Influence of traffic conditions on driver behavior before making a right turn at an intersection: Analysis of driver behavior based on measured data on an actual road. Transportation Research Part F: Traffic Psychology and Behaviour, 10(5), 397–413. Summala, H. (1996/0). Accident risk and driver behaviour. Safety Science, 22(1–3), 103–117. Van Der Horst, A. R. A. (1990). A time-based analysis of road user behaviour in normal and critical encounters. Unpublished PhD Thesis, Delft University, Delft, the Netherlands. Van Winsum, W. (1998). Preferred time headway in car-following and individual differences in perceptual-motor skills. Perceptual and Motor Skills, 87 (3 PART 1), 863–873. Van Winsum, W., & Heino, A. (1996). Choice of time-headway in car-following and the role of time-to-collision information in braking. Ergonomics, 39(4), 579–592.
MOTORCYCLE ERGONOMICS: SOME KEY THEMES IN RESEARCH S. Robertson1 , A.W. Stedmon2 , P.D. Bust3 & D.M. Stedmon2 1
2
Centre For Transport Studies, UCL, London, WC1E 6BT, UK Centre for Human Motorcycle Ergonomics & Rider Human Factors, University of Nottingham, UK 3 Civil and Building Engineering Department, Loughborough University, UK
The motorcycle and it’s user provide a unique example of a system that encompasses a wider range of ergonomics issues than are generally encountered in one place. This niche area of ergonomics has generated a large amount of interest and, as a result, a special interest group (SIG) was founded in 1999. The SIG has provided a point of contact for people interested in motorcycle ergonomics from around the world. This paper reflects upon a number of key themes in research within motorcycle ergonomics and provides a brief overview of some of the latest findings from a series of studies undertaken over the last 25 years.
Motorcycle design, style and control In this paper the term ‘motorcycle’ refers to two wheeled motor vehicles (TWMVs) which include conventional motorcycles as well as mopeds and scooters. There are variations in the legal definition of a motorcycle around the world and, unless otherwise stated, this paper considers the UK. In many ways, modern motorcycles have not changed much since they were first invented over 100 years ago. In the post war years of the 1940s to 1960s motorcycles evolved as functional designs that provided a cheaper alternative to the car. With the rise of affordable cars, the motorcycle started to become a leisure product as well as a means of transport. Today, more than half of motorcycle usage in the UK is still for work or education related purposes (DfT 2007). Trends in motorcycle ownership and in use do not always follow predictable patterns and Jamson, et al. (2005) provide a detailed discussion of the changing trends usage in the UK over the last decade. The operation of a modern motorcycle is substantially different from other motorised land transport in the way that controls are usually operated: • • • • • •
the clutch is a lever, operated by the left hand the front brake is a lever, operated by the right hand the throttle is a twist grip operated by the right hand the gearbox is operated by a lever using the left foot the rear brake is a lever, operated by the right foot lights, indicators, horn and ancillary equipment are usually operated by the fingers without removing the hands from the hand grips 432
Motorcycle ergonomics: Some key themes in research
433
Stedmon (2007) noted that riders tend to adopt typical postures (e.g. ‘sit up’, ‘lean-forward’, ‘lean-back’) when riding and to an extent this is dependent on the type of motorcycle being ridden, the clothing the rider wears and the speed which they are riding, but is more generally governed by the relationship of the seat height, handlebar height and position of the footrests. Classifications of motorcycles have been made (Robertson, 1986; Robertson & Minter, 1996) which included tourer, commuter, sport, trail/trial, stepthrough, custom and combination (sidecar) but as the styling and use of motorcycles has changed it has been necessary to revise them (Stedmon, 2007). It is important however, that newer definitions can take account of older styles so that longitudinal comparisons can be made. Until recently motorcycle controls were not very standardised. In the 1950s some machines still had hand operated gears; in the 1970s scooters frequently had gears operated by a left hand twist grip. Motorcycles built in different countries often had foot operated gears operating in different directions and/or located on different sides of the motorcycle. This lack of standardisation has not been very ‘user-friendly’ and whilst this is less of an issue on modern motorcycles there are still differences in the way manufacturers design control mechanisms such as indicators etc. Whilst some motorcycles have advanced anti-lock braking systems and even automatic gearboxes, in most cases the controls are simple cable or hydraulic mechanisms. However, in the last few years ‘ride by wire’ technologies have been developed that are slowly arriving in the marketplace. In addition, motorcyclists have begun to adopt information and communication technologies such as satellite navigation systems and wireless intercoms for rider and pillion as well as bike-to-bike communications. With these developments in motorcycle design, understanding the way users interact with the motorcycle itself and the wider road environment is fundamental to motorcycle ergonomics. A reflection of the growing importance of motorcycle ergonomics over the last 25 years, was the founding in 1999 of an Ergonomics Society special interest group (SIG). Ergonomics research is now generating more interest and support from manufacturers, road safety organisations, motorcycle media and riders themselves.
A review of some key areas of motorcycle ergonomics Whilst many areas of transport ergonomics research have developed substantially over the last 20 years, motorcycle ergonomics has not expanded in the same way. One reason could be that motorcyclists represent a niche road user group in many countries and resources may have been focussed on other higher profile or profitable initiatives. Another possibility is that in a highly competitive market with only a few major manufacturers a lot of research is commercially sensitive and therefore not in the public domain. However, with the expanding economies in the Far East who rely more on motorcycle transport than the West, things could change in the future (Stedmon, et al., 2008a). For example, there are high levels of motorcycle ownership in Taiwan with 0.48 motorcycles per person (Choua & Hsiaob, 2005) than in the UK where ownership is 0.021 motorcycles per person (DfT, 2007).
434
S. Robertson et al.
Table 1.
Distribution of motorcycle ergonomics publications (%).
Decade
Safety
Comfort
General ergonomics
1980s 1990s 2000s
40 25 28
40 50 36
20 25 36
The first explicit mention of motorcycle ergonomics identified by the authors was Lecomber (1974) however it was not then until the 1980s that the late Alfred Minter introduced ergonomics into the popular UK motorcycling press by describing aspects of the thermal environment (Minter, 1986). Since then, there has been increasing interest shown in ergonomics by the motorcycle media as well as in research publications. A database of approximately 80 studies related to motorcycle ergonomics over the last 25 years (represented in Table 1) illustrates that there has been a steady rise in publications and specifically those related to general ergonomics issues. Since 2000 there has been a fourfold increase in the number of studies compared to the 1990s. Within the identified ergonomics issues, anthropometrics has been a key area of interest but research has also investigated postural analysis of motorcyclists, computer modelling and the design of simulation equipment. Motorcycling represents a range of ergonomics issues including: constrained workstations, sociotechnical systems, physical environment (including vibration and noise), thermal comfort, rider clothing, comfort, posture and anthropometrics. Each of these is summarised below.
The motorcycle as a constrained workstation The concept of a motorcycle as a constrained workstation was first identified by Robertson (1986). Motorcycles are inherently unstable when stationary (the rider needs to actively keep it balanced or supported), but when moving, even at a low speed, they may only require minor inputs from the rider to maintain stability. Generally both hands need to remain on the handlebars to operate the controls so once on the machine and moving the rider has little opportunity to change their position and is constrained to almost a single static position. This postural fixity can lead to discomfort on long journeys. The relationship between rider size and position on the motorcycle can affect both the actual and perceived behaviour of the motorcycle as well as comfort and so it is often necessary for the rider to fit the machine within a small range of tolerances. The styling of motorcycles is an important factor. The popular motorcycling press (Hargreaves, 2005) has suggested that, between 1993 and 2003 sports motorcycles lost approximately 17 kg in weight. With an emphasis on a race styled forward-canted riding position, motorcycle design has resulted in more weight being transferred forward with an increase in seat heights. Of the motorcycles considered, seat heights have risen by approximately 6 cm in the last 25 years with the result that shorter riders find it more difficult to place their feet on the ground. To
Motorcycle ergonomics: Some key themes in research
435
date there has not been a fully systematic investigation into this aspect of changes to motorcycle design and the underlying ergonomics issues.
Motorcycles as part of a socio-technical system One of the fundamental premises of contemporary ergonomics is that people do not interact with things in isolation but within the context of a wider system. With road vehicles there are many aspects common to their use in the context of the road system (e.g. Helander 1976). The motorcycle and its rider do, however, have fundamentally different person-machine interfaces and can also interact with the road and traffic system in different ways from other motor vehicle types (Martin, et al., 2001, Robertson 2003, McInally, 2003, Lee, et al., 2007). In particular a review of a number of studies indicate that motorcyclists appear to have faster braking responses than car drivers, but that their braking capability is less than that of cars (Lee, et al., 2007). They also utilise the road space in a different way to other types of vehicle. What other road users may perceive as risky or aggressive behaviours may be due to the differences in the capabilities of motorcycles and other vehicle types. This is a clear example of the importance of looking at physical constraints and capabilities of motorcycles as well as the behaviours exhibited by their users. Whilst many aspects of motorcycle design can be studied as discrete studies (such as rider comfort, anthropometry, rider behaviour, etc) they only remain partial investigations if their place in the socio-technical system is not considered.
The physical environment By their nature, motorcycles leave riders exposed to the environment (e.g. weather and wind). Stedmon, (2007) noted this can affect the stability and safety of the motorcyclist and ultimately the effort it takes to control the motorcycle in different conditions. To put the environment into context, the rider is out of doors and will be exposed to sun, rain and wind. The wind will be the vector sum of the velocities of the motorcycle and the ambient wind. Driving at 35 mph on a still day the rider is exposed to a wind described on the Beaufort scale as a moderate gale whilst at 70 mph it would be described as a violent storm. This is further compounded if it is raining. A further issue relating to wind is the level of noise generated within a motorcycle helmet which is particularly important in the context of Noise at Work regulations. Lower, et al. (1996) reported levels of 90-109dBA at simulated speeds of 50–70 mph and were also able to make some recommendations for solutions. The rider may also experience turbulence generated by other vehicles or by the sudden changes in ambient wind strength. In terms of the control of lateral position on the road, constant wind can be easier to compensate for, but sudden lateral gusts are more likely to cause problems. At higher speeds and with some types of motorcycle, turbulence can shake the body and head and this may contribute to potentially higher cognitive loads. The environment can also expose riders to vibration from the engine and the road surface which is often transmitted through the handlebars, footrests or frame of the motorcycle to the seat. Rider comfort studies have identified vibration as a key issue with 22% of riders suffering discomfort in this way (Robertson, 1986). Mansfield & Irving (2001) reviewed work on motorcycle seat
436
S. Robertson et al.
vibration and reported on a tool for measuring motorcycle seat vibration. Research into frequency response functions of two wheeled vehicles on asphalt roads has enabled the calculation of a numeric comfort index for a particular vehicle on a set journey (Cossalter, et al., 2006). Again the amount of research into these niche areas seems limited.
Thermal comfort The rider faces a hostile outdoor environment with rain and ambient temperatures which can range during the course of a year from below freezing to over 30 degrees Celsius. Indeed, temperatures can range dramatically on even a short ride as ambient temperatures and wind chill (from vehicle speed as well as ambient sources) as well a radiant heat from the sun can impact on the thermal comfort of a rider. Motorcycles therefore represent the most extreme form of road transport given the thermal effects on the user. In the course of a few minutes or miles along the road a rider may experience such extremes as over-heating at low speeds and/or in warm weather or the risk of hypothermic symptoms in cold weather and/or sustained higher speed riding. Woods (1986) described the thermal problems of motorcyclists empirically and quantified the effects of a variety of measures to keep riders warm. For example, the addition of a full fairing to a motorcycle was equivalent to an increase in ambient temperature of 9 degrees Celsius.
Clothing In the UK, riders are only required by law to wear a helmet and so any other protective clothing is a matter of personal choice. Many motorcyclists will wear protective jackets, trousers, gloves and boots and this clothing can affect the quality of the interaction the rider has with the motorcycle as heavy gloves can reduce tactile feedback of controls; tight clothing can make it difficult to twist the body to make rear observations (Stedmon, 2007). In some weather conditions, remaining dry and at an appropriate temperature becomes a challenge and clothing provides protection not only against abrasion but also it provides thermal protection. Research has been undertaken to assess the performance of clothing (e.g. Mansfield, 2008) and the effects on the rider (Woods, 1983; 1986) where different types and combinations of windproof clothing made a difference of 3 to 6 degrees Celsius to the effective ambient temperature. In many cases research is commercially sensitive and the details are not always available in the public domain.
User expectations Within product design, the pleasure that is derived by users is an important concept (Jordan, 1998). With the potential dangers, vulnerability, the impact of poor weather conditions and the lack of a user-centred approach to meeting many user needs, it is perhaps the degree of pleasure and emotional bonding that defines the popularity of motorcycle use in the leisure sector. Furthermore, the ability of motorcycles to permeate traffic jams and short, consistent journey times makes them attractive for the commuting and some types of rapid response work such as the emergency services. As early as 1902, reports in the earliest magazine publications for motorcycles
Motorcycle ergonomics: Some key themes in research
437
compared the luxurious nature of the motorcar with the pleasurable riding experience of the motorcycle Koerner (2007). In some cases, the pleasure that a product may bring through its style or status may override its functionality. There is a possibility that this may be happening in the leisure motorcycle market in the UK with sports bikes taking up the largest market sector. In many cases the styling of these motorcycles and the ‘lean-forward’ posture riders should adopt, does not suit the typical use of these motorcycles for short commuting activities where a more upright posture allows riders to observe the traffic around them. As a result sports bikes are often considered (and even expected to be) uncomfortable. Journeys for leisure purposes do, however account for less than half of the journeys in the UK (DfT 2007) and the needs of those who are using motorcycles for work or travel to work must not be forgotten.
Rider anthropometry One area that has generated a lot of research is the anthropometry of motorcyclists. Robertson (1986, 1987) first reported on UK riders with self-report data in which it appeared that motorcyclists were taller than the general UK population. In 1993 this work was followed up (reported in Robertson and Minter, 1996) with an anthropometric survey of 140 motorcyclists which found that stature was significantly greater than the general UK population. More recently, a study in 2007 (reported in Stedmon, et al., 2008b) and also in 2008 using a directly comparable methodology were conducted. While the figures shown Table 2 initially suggest that the stature of riders and the general UK population is converging, further analysis is in progress looking at possible age effects of the sample. Choua & Hsiaob (2005) also demonstrated the use of anthropometric data together with a systematic assessment of comfort in the design process of scooters in Taiwan. This study illustrated the importance of identifying the intended user population and obtaining the appropriate measurements to ensure that the design solution ‘fits the rider’. The population were scooter riding students, the data were mixed gender and so it is Table 2.
UK motorcyclist and general population stature data (since 1981).
Source
Population
1981 data OPCS (1981) 2006 data The Information Centre (2008) 1986 data Robertson (1987) 1993 data Robertson and Minter (1996) 2007 data Stedmon, et al. (2008b) 2008 data Stedmon & Stedmon
General UK population General UK population UK motorcyclists (self reported) UK motorcyclists UK motorcyclists UK motorcyclists
Male Female N, mean (sd) mm N, mean (sd) mm 4719 1738 (67) 6116 1752(1.1) 109 1772 (62.9) 108 1774.0 (82.4) 48 1760.6 (82.5) 40 1762.0 (81.4)
5167 1609 (61) 6416 1616(0.9) 10 1645 (94.1) 31 1639.7 (89.4) 11 1650.0 (88.1) 11 1636.8 (46.5)
438
S. Robertson et al.
not possible to draw comparisons with data for the general Taiwanese population or the UK population.
Rider comfort Rider comfort is a key factor in motorcycle usability. A number of studies using comfort ratings have indicated patterns of discomfort associated with particular types of motorcycle (particularly between sports bikes and road bikes vs tourers) in relation to aches & pains and thermal issues (Robertson, 1986, 1987). A broad pattern of discomfort has been observed for both UK riders (Robertson 1986, Stedmon, 2007; Mansfield, 2008) and Taiwanese riders (Choua & Hsiaob, 2005) which illustrate that wrists are often rated with the highest discomfort ratings (other common areas include buttocks, lower back, upper back, upper arms, neck, and hands). The incidence of discomfort has also been related to stature as Robertson (1986) noted that taller riders had different patterns of discomfort compared with those of shorter riders. These patterns of discomfort reflected riders’ comments on aspects of the riding package that needed modification (e.g. short riders tended to want lower seats). There is a large body of work in research related to the comfort of the riders wrists (Robertson 1986, 1987; Stedmon, 2007) this is partly due to the size and posture of the rider but also is related to the hand operated controls on the handle bars. Some manufacturers are starting to take this on board and Marklin (2008) has indicated that muscle fatigue in the context of clutch lever use has led to successful engineering solutions aimed at reducing clutch pull weight. Horberry, et al. (2008) in a review of rider fatigue identified the general lack of research in the area, but did indicate that discomfort, thermal and postural stress could contribute to fatigue with its associated impacts on performance. As one of the main uses of motorcycles is to commute to work (DfT, 2007), it is also important to consider the nature of motorcycle riding combined with any physical loading of work activities which may result in discomfort experienced either on the motorcycle or in an occupational setting (Bust, 2002).
Motorcycle modifications Unlike other prevalent forms of road transport which have adjustment of the workstation designed into the vehicle, the users of motorcycles often require additional or modifications of parts to replace standard components. Quite often these are simple features such as heated-grips or aftermarket throttle kits to reduce the amount of wrist twist required to fully open the throttle. On the basis of selfreported data, about 1 in three people modified their motorcycles where (82%) of these modifications were to improve comfort or ease of use and 9% were to improve handling. Only 2% made changes to affect performance (Robertson, 1986). It is not economically viable for many manufacturers to supply a full range of components and so niche manufacturers or even the end users themselves have to develop their own solutions. This has important implications for policies such as strict type approval which could reduce the options for riders to improve the comfort and hence safety of their motorcycles.
Motorcycle ergonomics: Some key themes in research
439
Future research From the review of motorcycle ergonomics a number of issues arise. One of the key areas is the development of a dialogue between the ergonomics community and the motorcycle industry so that motorcycle designers and manufacturers can incorporate user requirements in greater detail within the design process. For some types of machine and rider, the benefits of image and pleasure aspects of a design may outweigh the associated reduction in functionality but in many cases it would appear that making the machine fit the rider would expand existing markets or open up new markets. These aspects need further investigation. The motorcycle and rider in the context of the road system would make a particularly good example upon which to test models of or approaches to the science of ergonomics. Longitudinal studies are often useful in order to help understand the user needs. A survey of the physical characteristics of current and of older motorcycles and motorcyclists is required. This could be used to develop a database from which physical dimensions could be extracted to consider the relationship between motorcycle design and user experience. In many studies respondents have been asked to identify the type of bike (make and model or a classification) they ride, but it would be too complex to ask them to measure their own bikes in a standardised manner. A tool to help the mapping of user descriptions or makes/models on to a classification scheme would be a great benefit for longer term research work. In terms of rider anthropometry, the existing work has already given strong indications about the impact of physical size on motorcycle usability, but information about smaller mopeds and scooters is limited. The authors suggest that a useful way of progressing this work would be to undertake a large planned sample of motorcyclists in a broadly similar manner to that used for car drivers by Haslegrave (1980).
Acknowledgements & dedication This paper is dedicated to the late Alf Minter (1922–2008). Many thanks go to the people who have provided information and insights into their various researches through personal communications. As this paper represents a review of motorcycle ergonomics, a subject area that led to establishment of a SIG, the authors are grateful for the small extension on page limit for this paper.
References Bust, P.D. (2002) Developing and ergonomic tool to assess occupational use of motorcycles. In, P.T. McCabe (ed) Contemporary Ergonomics 2002. Taylor & Francis: London, 289–293 Choua, J.R. & Hsiaob, S.W. (2005) An anthropometric measurement for developing an electric scooter. International Journal of Industrial Ergonomics 35(11), 1047–1063
440
S. Robertson et al.
Cossalter, V. (2006) Motorcycle Dynamics (2nd Edition), Lulu (http://www. lulu.com) Cossalter, V., Doria, A., Garbin, S., & Lot, R. (2006) Frequency-domain method for evaluating the ride comfort of a motorcycle, Vehicle System Dynamics, 44(4) 339–355 DfT (2007) Compendium of Motorcycle Statistics. Department for Transport: London [online] Accessed 14/11/08 (http://www.dft.gov.uk/pgr/statistics/ datatablespublications/vehicles/motorcycling/motorcyclingstats2007) Hargeaves, S. (2005) Wanted: A sportsbike for big blokes. ‘Bike’ Magazine (May 2005). EMAP: Peterborough. 83–88 Haslegrave, C.M. (1980) Anthropometric profile of the British car driver. Ergonomics 23(5), 437–467 Helander, M. (1976) Vehicle Control and Highway Experience: A Psychophysiological Approach Chalmers University of Technology, Division of Highway Engineering Horberry, T., Hutchins, R., & Tong, R. (2008) Motorcycle Rider Fatigue: A Review. Road Safety Research Report No.78. Department for Transport: London [online] Accessed 1/12/2008 http://www.dft.gov.uk/pgr/roadsafety/research/rsrr/theme2/ riderfatigue.pdf The Information Centre (2006) Health Survey for England 2006 Latest Trends. [online]Accessed 14/11/08 (http://www.ic.nhs.uk/statistics-and-data-collections/ health-and-lifestyles-related-surveys/health-survey-for-england/health-surveyfor-england-2006-latest-trends) Jamson, S., Chorlton, K., & Conner, M. (2005) The Older Motorcyclist. Road Safety Research Report No.55. DfT: London [online] Accessed 1/12/2008 http:// www.dft.gov.uk/pgr/roadsafety/research/rsrr/theme2/theoldermotorcyclistno 55.pdf Jordan, P.W. (1998) Human factors for pleasure in product use. Applied Ergonomics 29(1) 25–33 Koerner, S. (2007) Whatever happened to the girl on the motorbike? British women and motorcycling, 1919 to 1939, International Journal of motorcycle studies. March 2007 [Online] Accessed 15/11/2008 (http://ijms.nova.edu/March2007/IJMS_Artcl.Koerner.html) Lecomber, B. (1974) Sitting pretty. Motorcycle Mechanics (November 1974) 44–45 Lee, T.C., Polak, J.W., & Bell, M.G.H. (2007) Do motorcyclists behave aggressively? A comparison of the kinematical features between motorcycles and passenger cars in urban networks. In, Proceedings of 11th World Conference on Transport Research, Berkley 2007, Berkeley, USA [online] Accessed 1/12/2008 (http://www.tsc.berkeley.edu/newsletter/fall2007/C3/1400/ wctr_submit20070501.doc) Lower, M.C., Hurst, D.W., & Thomas, A. (1996) Noise levels and noise reduction under motorcycle helmets. in Proceedings of Inter-Noise 96, Hill F.A and R. Lawrence R (eds), Inst. of Acoustics, St. Albans, UK, 979–982.[online] Accessed 1/12/2008 (http://www.isvr.co.uk/reprints/inter96mc.pdf) McInally, S. (2003) R3 – A riding strategy formulation model for the risk averse motorcyclist. In, P.T. McCabe (ed) Contemporary Ergonomics 2003. Taylor & Francis: London 423–428
Motorcycle ergonomics: Some key themes in research
441
Mansfield, N.J. (2008) Personal communication Mansfield, N.J., & Irvine, J.P. (2001) A development of a seat pad suitable for measuring motorcycle seat vibration. Presented at the 36th United Kingdom Group Meeting on Human Responses to Vibration, held at Centre for Human Sciences, QinetiQ, Farnborough, UK, 12–14 September 2001 Marklin, R. (2008) Personal communication Martin, B., Phull, S., & Robertson, S. (2001) Motorcycles and congestion. In, Proceedings of the European Transport Conference. Published as a CD Association for European Transport, PTRC Volume Ref: ETC2001. Available from PTRC Training for Transport (http://www.ptrc-training.co.uk) Minter, A. (1986) The chill factor: How to keep warm on a motorcycle in deep midwinter. Motorcycle Rider: The Journal of the British Motorcyclists Federation 20(2) BMF:New Malden. 17–19 OPCS (1981) Adult heights and weights survey, OPCS Monitor Ref SS81/1 Office of Population Censuses and Surveys: London Robertson, S. (1986) An Assessment of the Ergonomic Problems of Motorcycle Riders with Special Reference to the Riding Position and Seat Height. MSc Thesis: Loughborough University Robertson, S. (1987) Exploratory Motorcycle Ergonomics. Transport Studies Unit: Oxford University (TSU ref 406) Robertson, S., & Minter, A. (1996) A study of anthropometric characteristics of motorcycle riders. Applied Ergonomics 27(4) 223–229 Robertson, S. (2002a) Motorcycling and congestion: Definition of behaviours. In McCabe P.T. (ed) Contemporary Ergonomics 2002. Taylor & Francis: London. 273–277 Robertson, S. (2002b) Motorcycling and congestion: Quantification of behaviours. In P.T. McCabe (ed) Contemporary Ergonomics 2002. Taylor & Francis: London 278–282 Robertson, S. (2003) Motorcycling and congestion: Behavioural impacts of a high proportion of motorcycles. In, P.T. McCabe (ed) Contemporary Ergonomics 2003. Taylor and Francis: London 417–422 Stedmon, A.W. (2007) Motorcycling is a pain in the neck, numb bum, aching wrists – or all these things! A rider comfort survey. In, P. Bust (ed) Contemporary Ergonomics 2007. Taylor & Francis Ltd: London 127–132 Stedmon A.W., Brickell E., Hancox M., Noble J., & Rice D. (2008a) ‘MotorcycleSim’: a user-centred approach for an innovative motorcycle simulator. In D. Golightly, T. Rose, J. Bonner, & S. Boyd Davis Creative Inventions & Innovations for Everyday HCI (Proceedings of Create2008, 24–25 June 2008) The Ergonomics Society: Leicester 37–44 Stedmon, A.W., Robertson, S., Yates, T., & Fitzpatrick, B. (2008b) Profiling a new user group: Anthropometric data for motorcyclists. In P. Bust (ed) Contemporary Ergonomics 2008. Taylor & Francis Ltd: London 779–784 Woods, R. (1983) The relationship between clothing thickness and cooling during motorcycling, Ergonomics 26(9) 847–861 Woods, R. (1986) The relationship between clothing thickness and cooling during motorcycling. Ergonomics 29(3) 455–462
KEEPING IT REAL OR FAKING IT: THE TRIALS AND TRIBULATIONS OF REAL ROAD STUDIES AND SIMULATORS IN TRANSPORT RESEARCH A.W. Stedmon1 , M.S. Young2 & B. Hasseldine1 1
Human Factors Research Group, University of Nottingham, Nottingham, UK 2 Human-Centred Design Institute, Brunel University, Middlesex, UK Whilst real world data is particularly valuable in understanding complex issues within transport research, it is not always possible to conduct research which may compromise safety. As a consequence, simulators are often developed and employed to bridge the gap between what is possible in the real world and what is not. This paper is presented in two parts: a consideration of the relative merits of simulator studies and the issues associated with developing simulators for transport research; and a number of case study examples of research the authors have been involved with reflecting on pragmatic aspects of real road and simulator research. Attention is given to the different research opportunities, methodological requirements, challenges and limitations that each context provides.
Simulators for transport research Collecting real world data is extremely valuable and underpins ergonomics as an applied discipline. Within transport research real world data provides a unique insight into human interaction within the real context of use. However, ‘keeping it real’ is not always practical, or indeed ethical, especially when investigating user behaviour in situations which could compromise personal or public safety. As a result, simulators have emerged and continue to be developed as a major research tool. ‘Faking it’ therefore allows road users to perform complex tasks under controlled conditions in the relative safety of a laboratory. Simulators offer a level of abstraction from the real world by providing an artificial environment in which users can experience characteristics of a real system. Historically they have evolved as training tools and technologically they offer solutions ranging from part-task trainers to high-fidelity replica applications. However, a simulator does not have to consist of full replica equipment, high quality graphics, and motion systems, as is popularly conceived. Simulation can include non-technology applications (such as paper-based schematics and cardboard mock-ups, group work, or face-to-face role play) through to simple computer based desktop simulations, situation or equipment emulations and basic trainer systems. Simulations can also include real equipment used in a non-operational setting. Within aviation and maritime applications, simulators provide a cost effective resource to support training in both military and civilian applications. The aviation 442
Keeping it real or faking it
443
industry has been a main driver in the development of simulation technologies, with the earliest flight simulators being developed in the interwar years. More recently, the rail industry has invested in this technology. In the last 10 years there has been rapid growth in simulators for driver training following the recommendations of Lord Cullen after the Ladbroke Grove accident. Whilst many of the simulators in use by train operators are of similarly high fidelity, there are also part-task trainers and computer-based training systems in common use. Alongside training, transport simulators have also found a clear niche in research and development as powerful tools for investigating and modelling user interaction. Train simulators have been built and used to investigate rail infrastructure problems and changes such as signal design and the impact on drivers regarding cognitive issues, driveability and route safety (Yates, et al, 2007). Within road transport, car simulators have been used for research into driver behaviour using advanced speech recognition interfaces (Stedmon, et al, 2002), satellite navigation and driver information systems (Pettitt, et al, 2007; Burnett, 2008) and the evaluation of conceptual vehicle control systems (Young & Stanton, 2004, 2007a). A new development in transport research is the focus on motorcycle ergonomics and recently a dedicated motorcycle simulator ‘MotorcycleSim’ has been built. Using MotorcycleSim it is possible to put riders through identical scenarios which are not possible on the road (where even lighting, traffic, weather conditions can vary on the same route between different rides). This means it is possible to control many of the variables in the real world interactive system and rigorously investigate factors independently. Simulators also allow riders to experience demanding road conditions where they can make mistakes and test the limits of their riding ability in the safety of the laboratory (e.g. riding in hazardous conditions or to test endurance, alcohol effects, rider distraction, etc). For more information on MotorcycleSim please refer to Stedmon, et al, (2008).
Research issues in simulation As with many areas of empirical research, debate over the relative merits of laboratory and field research is contested within transport research. Whereas real world research offers greater ecological validity, it suffers from a lack of experimental control over many variables due to the highly dynamic context in which the user interacts. Transport simulators offer precise control over extraneous variables (such as weather, traffic, and road conditions) that might otherwise introduce error into a field study. Simulators also provide an opportunity to model hazards that are not easy to recreate or capture in real road trials due to their infrequency and unpredictability. Moreover, these variables are consistently repeatable to ensure all participants experience the same manipulations under identical experimental conditions. On a more practical level, it is possible to run simulator trials regardless of the weather or variable traffic conditions on real roads, and it is also possible to abort simulator scenarios immediately, which is not so easy in the real world. From an economic and ecological perspective simulator studies typically consume fewer resources than on-road research (although the capital and maintenance costs of a simulator are not inconsiderable).
444
A.W. Stedmon, M.S. Young & B. Hasseldine
Simulators also offer an opportunity to employ a range of metrics which would be precluded on practical or ethical grounds for a real road study. For instance, studies on driver mental workload used a secondary task to assess spare attentional capacity, involving drivers dividing their resources between driving and an additional task (Young & Stanton 2004, 2007b). Methods such as this and the use of head-mounted eye-tracking equipment are inadvisable in real road studies for safety reasons. Real world methods usually focus on pre-trial and post-trial comparisons or measures have to be non-intrusive (e.g. psycho-physiological or remote monitoring using video analysis). Furthermore, simulation software usually allows for the capture of much more accurate data on a wealth of driving performance measures than is practical in all but the most advanced instrumented vehicles. From a safety perspective participants can encounter situations in a simulator which would be unethical in the real world (for instance, where participants are able to crash or drive dangerously without any risk to their health or injury to others). With simulators users are able (and sometimes even encouraged) to make mistakes that would not be possible in a real scenario on the road. Simulators, therefore, provide an opportunity to address fundamental questions about user safety, risk and hazard perception, cognitive distraction. Research has illustrated that detrimental effects occur when car drivers eat and drink whilst driving (Young et al, 2008) and simulator trials have also highlighted important distraction effects from roadside advertisements which affect driver safety (Young & Mahfoud, 2007).
Simulator fidelity and validity Whilst simulators provide a safe environment for research this can be a potential weakness from a methodological point of view as they can suffer limitations in fidelity and validity. In a review of simulator use two dimensions of fidelity were considered to have the greatest impact on the validity of simulator research: physical fidelity (the extent to which the simulator looks like the real system) and functional fidelity (the extent to which the simulator acts like the real system) (Stanton, 1996). In contrast to simulators for training, it has been argued that physical fidelity in research simulators can be compromised without affecting the transferability of results as long as functional fidelity is maintained (Stammers, 1986). There would appear to be a law of diminishing returns associated with physical fidelity, such that vast increases in cost of a simulator may only lead to small increases in the quality of data obtained (Stanton, 1996). Nonetheless, physical fidelity may promote characteristics of immersion and involvement, which are prerequisites that support the user’s experience of ‘presence’ and thus the degree to which they might take the simulation seriously and behave in a realistic manner (Stanney & Salvendy, 1996). Fidelity is a fundamental issue in simulator research which underpins the validity of a study and the transfer of findings to the real world. Modern computing power means that high levels of functional fidelity and physical fidelity can be achieved for relatively low investment. However there is a danger that physical fidelity might be over-specified when it might be better to use limited or part-task simulators (Rushby, 2006). It is usually the physical fidelity (the hardware of the simulator) that incurs
Keeping it real or faking it
445
the greatest financial set-up costs and one aspect commonly associated with this is physical movement of the simulator. Motion systems can enhance the face validity of a simulator for participants and external observers if implemented effectively. However, if the motion cues are inaccurate it can seriously detract from research purposes and give rise to simulator sickness. This is a phenomenon with similar symptoms but different aetiology to motion sickness and affects approximately 5% of the population (Kennedy et al, 1993). In real road research there is potential for participants to experience symptoms of general discomfort or motion sickness and so this is not just restricted to simulator studies. It is important, therefore, to have a clear focus when developing simulators so that they do not become a slavish attempt to recreate the real world system, but, more importantly, serve the purpose for which it is intended (training, research, development, etc) with a fundamental understanding of user requirements.
Taking account of user requirements for simulators Whilst many areas of transport simulation have developed over the last 20 years there has been a notable exception with motorcycles and as a result motorcycle simulation technology is virtually non-existent (Stedmon, et al, 2008). With the development of a dedicated motorcycle simulator it was important to capture user requirements from the outset. Using a detailed user requirements capture methodology a number of variables were identified which were considered important for a motorcycle simulator including: leaning action like a real motorcycle; realistic acceleration/braking effects; realistic controls; realistic feel of the road from the motorcycle; feedback from the handlebars; sitting on a real motorcycle; countersteering facility; comfortable to use; engine/motorcycle vibration; different weather effects; easy to ride; sound effects in ear of rider; a good visual model; a facility to model other traffic; simulating rider with a passenger; looks good. The result is that ‘MotorcycleSim’ was designed as the world’s first full size, interactive motorcycle simulator that is linked to dedicated ‘STI-Sim DRIVE’ simulation software which provides realistic riding scenarios. In relation to the physical fidelity of the simulator, a real motorcycle is used which enhances the face validity of the simulator for users. The simulator is primarily a research tool and a preliminary study has been conducted to evaluate different aspects of the functional fidelity by testing different combinations of visual feedback and steering inputs to validate this against the user requirements (this is currently in preparation as a journal paper).
Case studies in real world and simulator research The authors have been involved in a number of studies which span real road research including experimental trials (Walker, et al, 2002) and naturalistic studies (Stedmon, 2008), quasi-experimental research (Stedmon, et al, 2008) and motorcycle simulation. In this section the studies are summarised and compared.
446
A.W. Stedmon, M.S. Young & B. Hasseldine
Cognition of motorcyclists and car drivers (Walker, et al, 2002) In this exploratory study a number of motorcyclists and car drivers were recruited to investigate any differences in rider or driver cognition. Three motorcyclists and twelve car drivers took part in the study (the car drivers were split into two groups of six according to the level of feedback their cars exhibited). All participants completed a 14 mile route on real roads and analysis was conducted on concurrent verbal protocols, situation awareness (SA) measures, and workload. The results illustrated that motorcyclists possess greater SA (possibly due to the close interaction they have with their motorcycles and the road environment), higher levels of verbal protocols (especially in relation to SA level 1 and 2) but also seemed to exhibit the lowest levels of workload (possibly because their verbal protocols enhanced their task performance rather than impacted on it). This study illustrated that real road research is possible but that it is not always easy to recruit large number of participants. This impacts on the power of the statistics and with only three motorcyclists involved in the study it is difficult to generalise the findings to a wider population. Real road research also poses problems with insurance of participants and risk assessments to make sure that potential health and safety issues are fully considered. This means that real road research can be particularly problematic to set-up and run but ultimately it’s the most ecologically valid as it is conducted in the real world.
Big Harted Bikers ‘World Tour of the UK’ study (Stedmon, 2008) This study followed 18 motorcyclists on a 1500 mile charity ride around the British Isles. The ride took place over 5 days and provided the potential to capture 90 rider days worth of data. Each day questionnaires were administered to assess rider workload, rider comfort and aspects of motorcycle design. In addition, the route was tracked by GPS and video capture techniques were used at various points throughout the study. Methodologically, this was a very cost effective way of collecting a large amount of data. As this was not primarily a research ride, there were occasions when it was difficult to motivate participants at the end of a long ride to complete questionnaires. As a result not everyone completed a questionnaire each day. The rider comfort ratings supported the general trend where the lower arms, neck, buttocks and particularly the wrists were affected by riding (Stedmon, 2007).
Quasi-experimental study of rider comfort (Stedmon, et al, 2008) This study employed a quasi-experimental approach where five motorcycle riders with different anthropometric characteristics rode eight different styles of motorcycles and provided comfort ratings across a number of dimensions. The riding trials took place on a dedicated test-track which allowed for greater control of the variables but was limited in terms of any realistic riding activities as there was no other traffic, street furniture, junctions, etc for riders to negotiate. The test-track meant that riders experienced the same conditions on each motorcycle (which was further enhanced as the weather remained constant throughout the trial) and so this
Keeping it real or faking it
447
provided a unique opportunity to investigate rider comfort on each motorcycle. The findings illustrated that ‘one size does not fit all’and different sized riders have very different experiences of the same motorcycle. It was also apparent that motorcycle riders experienced various degrees of discomfort when riding which also supported the general trend as reported by Stedmon (2007). From a practical perspective, running the trial over the course of a single day meant that riders reported feeling tired and bored at various times. Using the test-track involved complex logistics with everyone meeting at a remote venue and factors such as the cost of hiring the track would impact on the feasibility of doing this kind of research on a regular basis.
MotorcycleSim evaluation study This study was conducted to evaluate the functional fidelity of ‘MotorcycleSim’ by testing different combinations of visual feedback and steering inputs. Steering (positive-steering or counter-steering) and visual lean angle (no lean or 25 degrees of lean) were manipulated in a 2×2 within-subjects experimental design, which provided 4 conditions. 16 participants took part in the study which lasted 90 minutes. Data were collected for rider demographics and rider behaviour/attitudes, workload, comfort, pre- and post-simulator perception measures, simulator configuration preferences as well as objective data from the simulation software. The results indicated that 83% of riders preferred a tilting visual scene, 81% preferred positive-steering and 88% of riders considered the scenarios as realistic and relevant to real road conditions. In general, more accidents and violations occurred under the counter-steering conditions. As with the previous studies, rider comfort ratings followed the same pattern. This study illustrated that a range of different measures can be taken during one study, capitalising on the time that participants are involved in the study.
Comparing the different research approaches The different approaches illustrate varying degrees of research control and the potential to capture data within a specific timeframe: • individual real road trials – in general, these studies offer the greatest degree of realism (in terms of fidelity and validity) where riders may complete identical routes on a specific motorcycle for the study or their own motorcycle. Riders may ride alone (and therefore not be distracted or tempted to ride beyond their general abilities if following a group) but real world trials are vulnerable to ‘between-subjects’ inconsistencies (traffic, weather, etc) and can be difficult to set up and organise (risk, insurance, etc). Data capture can be slow as each ride is separate and it can be complicated to collect concurrent data whilst riding if it might cause a distraction to the rider and compromise their own or any other road user’s safety. • joining an organised ride – these rides are often pre-arranged by an external organiser and whilst this means that the logistics can be simplified, a degree of research control is sacrificed and it can be hard to motivate participants throughout the ride. The major strength of this approach is that it is possible to generate
448
A.W. Stedmon, M.S. Young & B. Hasseldine
many rider days of data in a relatively short time. There is a high degree of ‘between-subjects’ consistency as all riders take the same route at approximately the same time. This means that data is comparable across riders for the conditions they have ridden through collectively. This approach also lends itself to assessing riders over a number of days throughout a ride. A potential disadvantage is that riders in a group may ride differently compared to when they ride alone and so this needs to be taken into consideration when looking at rider behaviour. • quasi-experimental studies on a test track – this approach takes a more experimental perspective but uses the real world as the ‘laboratory’. As a result this approach allows a greater degree of experimental control in a more realistic setting on a test track. The normal hazards of the road are either taken out of the trials or controlled for and this provides the potential to capture many different measures in a single ride. These trials are usually short-term rides and so it is not possible to follow riders for days at a time. Also there can be considerable costs involved with hiring a test track for a specific trial and it can be difficult to coordinate participants on a single day. • simulator studies – these offer complete control over variables as the researcher builds the scenario to suit the research question. Data capture is as slow as any individual rider approach but ‘between-subjects’ variability is controlled for as participants undertake identical scenarios. This provides much more rigorous experimental control and statistical analysis of the data. This approach offers the greatest degree of safety as the simulator can be stopped at any time and scenarios can be manipulated to capture measures concurrently throughout a ride. It is also possible to use technologies that cannot be used on real motorcycles (e.g. heart rate monitors linked to computers, real time video capture of rider posture), but ultimately simulators suffer from issues of decreased fidelity and validity when compared to the real world applications they mimic.
The future of real road and simulator research The potential of transport ergonomics in all its guises is vast and both real road and simulations will feature on emerging research agendas in aviation, rail, and road research. To support real world research ‘instrumented vehicles’ will become more common. These are dedicated research vehicles that collect data on user behaviour and vehicle performance. Novel methods are likely to emerge (and are already in development) for capturing real world data in real time and in a nonintrusive manner. Simulated trials will provide a valuable research and training tool for users. Considering the factors underpinning the development of simulators, it could be argued that investment is better spent on software improvements (such as the simulator engine and the visual database) rather than hardware (once a certain level of fidelity is achieved). Ultimately, when considering real or simulation studies for research, there are a vast array of relative merits that need to be considered and balanced in order to arrive at an appropriate specification based on research needs and user requirements in terms of functional and physical fidelity. In some cases,
Keeping it real or faking it
449
that may mean choosing real road studies or quasi-experimental trials over simulator studies: its ultimately a case of using the right tool for the right job.
Acknowledgement The authors would like to thank Triumph Motorcycles, the British Motorcycle Federation, Phoenix Distribution (Arai helmets), ‘STI-Sim DRIVE’ and the ‘Big Harted Bikers’ for their support.
References Burnett, G.E. (2008) Designing and evaluating in-car user-interfaces. In, J. Lumsden (ed.) Handbook of Research on User-Interface Design and Evaluation for Mobile Technology, Idea Group Inc. Kennedy, R.S., Lane, N.E., Berbaum, K.S. & Lilienthal, M.G. (1993) Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness, International Journal of Aviation Psychology, 3(3), 203–220. Pettitt, M., Burnett, G.E. & Stevens, A. (2007) An extended keystroke level model (KLM) for predicting the visual demand of in-vehicle information systems. In: Proceedings of ACM CHI 2007 Conference on Human Factors in Computing Systems 2007, pp.1515–1524. Rushby, N. (2006) How will we know if games and simulations enable learners to learn anything? (Conation Technologies, Crawley). Stammers, R. B. (1986) Psychological aspects of simulator design and use. In J. Lewins and M. Baker (eds.), Advances in nuclear science and technology. (Plenum Press, New York). Stanney, K., & Salvendy, G. (1996) Aftereffects and sense of presence in virtual environments: Formulation of a research and development agenda, International Journal of Human-Computer Interaction, 10(2), 135–187. Stanton, N. (1996) Simulators: a review of research and practice. In N. Stanton (ed.), Human factors in nuclear safety (Taylor & Francis, London), 117–140. Stedmon, A.W., Richardson, J.R. & Bayer, S.H. (2002) In-Car ASR: Speech as a Secondary Workload Factor. In, P. McCabe (ed). Contemporary Ergonomics 2002. Taylor & Francis: London. Stedmon, A.W. (2007). Motorcycling is a pain in the neck, numb bum, aching wrists – or all these things! A rider comfort survey. In, P. Bust (ed). Contemporary Ergonomics 2007. Taylor & Francis Ltd. London. Stedmon A.W., Brickell E., Hancox M., Noble J., & Rice D. (2008) ‘MotorcycleSim’: a user-centred approach for an innovative motorcycle simulator. In D. Golightly, T. Rose, J. Bonner, & S. Boyd Davis Creative Inventions & Innovations for Everyday HCI (Proceedings of ‘Create2008’, 24–25 June 2008) The Ergonomics Society: Leicester. Stedmon, A.W. (2008) Riding for real world research. Motorcycle Rider, 40(4), 42–44.
450
A.W. Stedmon, M.S. Young & B. Hasseldine
Stedmon, A.W., Hargreaves, S., Fitz-Gibbons, M., & Wood, C. (2008) Different riders on different motorcycles: a quasi-experimental approach to rider comfort. In, P. Bust (ed). Contemporary Ergonomics 2008. Taylor & Francis Ltd: London. Walker, G.H., Stanton, N.A., & Young, M.S. (2001) An on-road investigation of vehicle feedback and its role in driver cognition: Implications for cognitive ergonomics, International Journal of Cognitive Ergonomics, 5(4), 421–444. Yates, T.K, Sharples, S.C., Morrisroe, G. & Clarke, T. (2007) Determining user requirements for a human factors research train driver simulator. In, J.R. Wilson, B. Norris, T. Clarke & A. Mills (eds.) People and Rail Systems. Ashgate: London. Young, M. S., & Stanton, N. A. (2004) Taking the load off: investigations of how adaptive cruise control affects mental workload, Ergonomics, 47(9), 1014–1035. Young, M.S., & Mahfoud, J.M. (2007) Driven to distraction: The effects of roadside advertising on driver attention. In P. D. Bust (ed.), Contemporary Ergonomics 2007, (Taylor & Francis, London), 145–150. Young, M.S., & Stanton, N.A. (2007) What’s skill got to do with it? Vehicle automation and driver mental workload, Ergonomics, 50(8), 1324–1339. Young, M.S., & Stanton, N.A. (2007b) Miles away. Determining the extent of secondary task interference on simulated driving, Theoretical Issues in Ergonomics Science, 8(3), 233–253. Young, M.S., Mahfoud, J.M., Walker, G.H., Jenkins, D.P., & Stanton, N.A. (2008) Crash dieting: The effects of eating and drinking on driving performance, Accident Analysis & Prevention, 40, 142–148.
SCHOOLS
PUPIL INVOLVEMENT IN CLASSROOM (RE)DESIGN: PARTICIPATORY ERGONOMICS IN POLICY AND PRACTICE A. Woodcock1 , J. Horton2 , O. den Besten2 , P. Kraft2 , M. Newman1 , P. Adey3 & M. Kinross1 1
2
Coventry University, UK University of Northampton, UK 3 University of Reading, UK
Over the last decade, UK policy interventions relating to children, young people and education have made pupil participation in the (re)design of school environments a key imperative. Indeed, pupil participation is an explicit, core ideal of major, ongoing school (re)construction and (re)design programmes. Pupil involvement in decision-making is – ostensibly – central to this commitment. This paper will present preliminary findings from a project exploring the possibilities for – and the present state of – pupil involvement in classroom (re)design and ergonomic decision-making. Through in-depth qualitative data drawn from pupils, school staff, Local Authority officers and other stakeholders we explore the relationships, and tensions, between the ideals of participatory ergonomics as expressed in national policy statements and the way such participation occurs in practice. These data suggest that the ideal of pupil participation may, in practice, be foreclosed by contingencies, budgets, issues, debates, personalities and events at grassroots level.
Introduction It is widely accepted that the design of the school environment has profound effects on the activities and outcomes of teaching and learning. School buildings should contribute positively to teaching and learning. As well as contributing to the academic wellbeing of students, the school should promote social interaction and a sense of community and inclusiveness. The Commission for Architecture and the Built Environment, which leads a campaign, entitled “Better Public Buildings” stated “. . . we know that good design provides a host of benefits. The best designed schools encourage children to learn.” (DCMS, 2000). An investigation by Price Waterhouse Cooper (2001), commissioned by the Department for Education and Skills found that there was a positive relationship between capital and performance and more specifically between the physical school environment and pupil performance. The report continues that “The general attitudes, behaviour and relationships amongst pupils and staff are more conducive to learning in those schools which have had significant capital investment”.
453
454
A. Woodcock et al.
Whilst acknowledging the importance of the design of the school environment on those who spend a large proportion of their time there (both teachers and children), UK (and worldwide) legislation has also recognised the need for children to be involved in the design of the spaces they inhabit. UK policy has taken steps towards (i) greater recognition of the diversity and distinctiveness of children and young people’s needs; (ii) a growing imperative for children and young people’s participation in decision-making and practices regarding their everyday lives and contexts. The UK Government has created three major lines of capital investment for school buildings (i) academy schools; (ii) Building Schools for the Future (BSF); (iii) the Primary Capital Programme (PCP). User involvement would seem to form a cornerstone of these initiatives, as evidenced in the following quote. “Putting the user experience at the heart of all we do is not the same as saying we simply respond to user demand. Our and our partners’ role is to prioritise, ensuring resources are used efficiently and effectively in the wider public interest. We need also to be aware that service users have very different capacities to shape services. . . . . . . The key lesson for taking forward the Five Year Strategy is this: failing to understand users in the way we design and deliver services means we are less likely to deliver aggregate improvements in outcomes across the system because we are less likely to be meeting the needs of individual service users” (DfES, 2006, p. 36). In addition to the statutory requirements, pupil participation is fundamental to the UK Government’s vision for all schools post-Every Child Matters: “There isn’t a design blueprint for a school of the future: a variety of models will emerge. The main design challenge facing LEAs and schools is to balance the needs of different users, creating inspiring buildings with functional spaces that are appropriate for new educational developments and new technologies but adaptable enough to cater for the changing needs of the future” (DfES/SBDU, 2002, p. 16). “[The user is] a key player in the success of a building project. It is very important that right from the beginning of a school building project there is proper consultation with the staff and pupils of the school and the wider community. The school and its community must decide what they need and want both for the immediate and longer term future. . . All potential users in the community should be consulted in order to assess local and individual needs. . . This approach will help to encourage greater use of the building, develop trust between all parties and add to the feeling of community and ownership. . . Consultation and feedback should continue through the construction period. Newsletters, websites and displays can all be used to update local users, clients and other interested groups on progress. Where work is taking place at an existing school, finding ways of linking the project to the curriculum can benefit pupils and encourage a positive attitude to the work taking place” (DfES/SBDU, 2002, p. 63).
Pupil involvement in classroom (re)design
455
In the AHRC project we are dealing primarily with the BSF programme, the guidance for which asserts that pupils and staff have to have an active role in the development of the design brief, down to the level of what is needed in each room (DfES, 2004a, p.6). However the guidance relating to the PCP (Primary Capital Programme) suggests the approach to participation may be more tokenistic than implied in the policy documents. From a critical reading of salient policy and guidance documentation, Horton (2007) hypothesises four key ways in which pupil participation may, in practice, be foreclosed in and through the UK Government’s three major schools capital investment programmes. First, there is mounting evidence that pupil participation can be limited because of the nature of public-private partnerships (PPP) in school construction because of the operational complexity and protracted duration of many PPP projects, and the proliferation of specialist, nationwide contractors offering relatively affordable, pre-prepared, generic, ‘off-the-shelf ’ solutions. To give one illustration, DfES (2004b) advisory notes regarding BSF outline the complex, protracted, iterative (9-stage) sequence of activities involved in constructing a school via PPP: (i) preparation; (ii) initiation; (iii) strategic planning; (iv) business case development; (v) procurement planning; (vi) procurement; (vii) contractual close; (viii) construction; (ix) operation. Indeed the DfES Building Bulletins warn of the difficulties of sustaining consultation and balancing the needs of a large group of stakeholders/interests in complex sequences of PPP activities, especially those which entail private finance and/or partnership with large corporate partners. DfES/SBDU (2002) guidance concluded that “it is essential to build time into. . . [a PPP] programme for users to discuss the proposals before they progress too far” (p. 67). Second, schools, like any public buildings, are subject to an array of Building Regulations, Health and Safety Regulations and British Standards. There are many aspects of school building/design which are rigidly, intricately and statutorily prescribed. This may mitigate against meaningful pupil participation. Third, moreover, for BSF, PCP and Academy Schools, there exists an array of standard, recommended specifications, layouts and dimensions for practically every element of the school estate. For example, the DfEE Managing School Facilities series specifies guidelines regarding practically every material aspect of a school estate. Likewise, DfES Standard Specifications, Layouts and Dimensions (SSLD) guidance notes for BSF projects (e.g. DfES, 2007a, 2007b, 2007c) lay out recommended standards – and a limited array of compliant design examples – relating to specific elements of BSF school buildings. Significantly, the only mention of pupil involvement in these SSLD guidance notes for BSF projects relates to the colour/decoration and upkeep of toilet cubicles! “Pupil involvement Involving pupils in the design process has been found to increase their sense of pride and ownership of their school buildings, helping to reduce vandalism and bad behaviour. Pupils should be involved in aspects of the design – such as choosing colours and finishes for the toilet
456
A. Woodcock et al.
facilities – wherever possible. Another option is to incorporate pupil artwork into the laminate finish of the cubicles and associated panels. Pupils may be involved in the management of the toilet blocks. For example, after peak periods of use school council representatives could check the toilets and report back any issues that need sorting out. It is essential if this is implemented that the problems highlighted by the pupils are dealt with” (DfES, 2007b, p. 14).
These guidelines are non-statutory, but they do indicate a limited array of detailed blueprints and ‘industry standards’. So, again, we might ask: to what extent is innovative, participatory design likely in such a deeply pre-specified context? Finally, there is the question of how participation might actually be achieved and sustained in practice. For while pupil participation is an integral aspiration within the programmess, there is very little practical guidance on how this may be achieved in practice in school contexts which are busy, overloaded and time-pressured. How and when should teachers ‘do’ participation, and moreover, how should they learn how to do it? This is compounded by the relative, and perhaps increased, isolation of schools as entities: for example, it is perhaps the case that the successive devolution of agency to schools themselves might mitigate against the sharing of participatory practices.
The scope of the project The 2 year AHRC funded project aimed to discover more about the practice of pupil participation in typical schools close to the two participating universities. Approximately ten schools were selected from Coventry and Northampton that varied in terms of their catchment area, specialism, stage in the redesign process and the scale of the redesign. The wider objectives of the project were to understand how pupils are involved in the school refurbishment design process, the type of contribution they can make, and how involvement in participant can affect wider curricular activity and understanding. This paper specifically addresses the first objective.
Methodology A qualitative approach was undertaken to understanding the situation in schools, with detailed ethnographic observations occurring in each school relating to usage and ethos. Where appropriate design consultation periods or design events were attended. These activities were supplemented by interviews with key stakeholders including the LEAs, architects and teachers. This paper presents preliminary findings from both types of study.
Pupil involvement in classroom (re)design
457
Specimen findings From interviews with LEAs 15 semi-structured telephone interviews were conducted with representatives of county/city councils in randomly chosen locations in England. These representatives held posts which required them to deal directly with school (re)building programmes. Thus, interviewees had roles such as Head of Planning and Building for Children’s Services, BSF/PFI Project Manager, School Site Development Manager, Head of Strategic Planning, Principal Development Officer, or Educational Assets Manager. These emphasize the connection between children and education, on the one hand, and asset development and building, on the other. Such officers work on projects for (re)building a particular school, cooperating closely with the school’s management team, architects and various consultants. Together, they develop a brief for the project and oversee it until completion. Thus, our assumption was that these professionals would be a link between government and local policies on children’s participation and the reality of school life in the context of a (re)building project. The idea of pupil participation was a familiar one to all participants and had sometimes already been embedded in the working ethos before government initiatives. However, the exact nature of the participation was seen as having to be controlled by the local situation i.e. it had to be unique, and tailored to the needs of the project. . . . [Y]ou never get two schools that approach things in the same way. There is a lot dependant on the way those schools were set up and the management structure and the involvement of the governors, relationships with staff and how involved the staff are, you never get two schools which are the same. We like to make sure they all have the opportunity to engage and involve as many people as possible but different schools have different ways of doing it. Therefore the nature of pupil participation was left to the school. ‘There is a different appetite in each school for this sort of activity [pupil involvement], whether or not it fits with the school ethos.’ Whilst the interviewees generally believed that pupil participation would reap numerous benefits, most admitted that in reality it had a rather limited character. Unfortunately and unexpectedly, pupil participation did not take place in a standard, regularised design process. The ideal, creative “hands-on” participatory design techniques (modelling, drawing etc.) were rarely mentioned by our interviewees, and if they were, it was not always in the context of pupils’ input to the actual design of the building, but rather when speaking about increasing their creativity and design awareness. Pupils were involved in school building projects in different ways and through different agents. Interviewees insisted that pupils are consulted. For example through consultation at a conceptual level, along with other stakeholders, pupils “fed into the vision side of things and that has altered some decisions . . .” As another local
458
A. Woodcock et al.
authority officer put it, ‘. . . it’s more about ‘bigger statements’, like ‘we want lots of light in the building’, ‘we need a place to store material’, ‘we need a space to play and be imaginative’ etc. Consultation can involve all pupils in the school for example, through a questionnaire; otherwise there is representative participation, when a selected group of pupils contributes eg ‘We asked schools to form ‘vertical consulting groups’ comprising staff, pupils, and parents. . . These groups, including pupils’ representatives, were on evaluating panel for our bidders’ ‘Involvement of children depends on [. . .] the attitude of the school towards involvement, in terms of time scale and type of project. Some might just work with the school council; in other schools there might be workshops of up to 80 children.’ Consultation may occur at the very first stage of the project, when, for example, the summarised answers from pupils’ questionnaires are fed into the initial brief, which the architects use as a point of departure; or later when pupils comment on the layout of the future building, prepared by the architects. It can also take place on both stages: ‘Quite often it happens like this: if architects already have some sketches and ideas, teachers and pupils can discuss them and leave their commentaries.’ The interviews showed that pupils are more likely to respond to certain issues such as dining spaces, toilet spaces, indoor and outdoor social spaces (such as playgrounds and common rooms). It is unclear whether their participation is limited to such areas because these are problematic and initiate a lot of debate, or because they are allocated these areas, as it is easier to organise a participatory design process around a limited area or issue. Either way, local authority officers insist that such partial participation is significant and has an impact on the final design. In one school, the head teacher got pupils to design toilets – it’s a very important area for them, however small. . . It depends which way you look at it. From our point of view, it’s a very small part of the scheme, but from the pupils’ point of view, it’s a huge part of the scheme because it’s the things, the areas we are removing are the parts that bother and trouble them so the areas that made it easy for bullying and intimidation are not there anymore [. . . ], then yes, then it’s a major impact. . . Another interviewee classified pupil participation as “on the margin – but an important margin”. Additionally consultation is sometimes substituted by advocating for pupils’ needs, especially in the cases when children are considered unable to actively participate in the process, either because they are too young or have a special needs condition: We’ve built mostly nurseries, early years schools, so children are too young to contribute to the process. But architects and other partners make sure that the children’s interests are observed – that the school is built for children and that the facilities are easy for them to use. . . In the end of the day, innovation that is shown on the final design reflects mostly teachers’ educational needs, but through them, those of pupils. . .
Pupil involvement in classroom (re)design
459
Another mechanism that has been revealed in the interviews can be called succession in participation: When we are replacing an existing school, we always try to involve pupils; otherwise, when it is a new school with no previous catchment, we use what we have found in previous schools: the design is not unique for each school, we are working from a set of ready designs, so in one way or another pupils’ ideas are reflected in them; By now, having done four schools already [. . .], we are quite experienced in this situation and therefore we are aware of some of the things that children are asking for. . . While consultation is a direct form of participation, advocating for children’s “spatial needs” in the school building and succession of pupils’ needs and ideas can be viewed as indirect participation. Key facilitators for participation were in most cases the architects. They had the most interaction with schools and also involved other partners in the life of the school. Organisations such as CABE, Extended Schools, and School Works and independent consultants were also mentioned. During the final stages of the project contractors liaised with schools and provided work experience to some of the older children. In summary the difficulties and barriers to pupil participation in design activity may be related to the complexity of the process, the building programme, the financial scheme, the pupils themselves and a lack of time and knowledge about how to control this process. In the final section of the paper we look at preliminary results from the observations in schools showing the way in which participation is experienced in schools.
From school observations The team spent extended periods of time in 11 schools, talking to staff and pupils, observing the design and use of the facilities and attending design meetings. These observations supported the interviews conducted with the LEAs concerning the individual nature and ethos of the schools, the varied nature of participation, the complexity and protracted nature of build programmes which means that consultation with the actual end user pupils of the building is unlikely and the array of issues. Taking as an example the experience of one primary school in our survey, Newman and Thomas (2008) related the following strategies used to engage students, such activities are typical of the ways schools engaged pupils: • consultation throughout the visioning process; • a ‘Design your school day’ in which gifted and talented students led groups of other students in designing and making a model of a chosen part of the new school; • PSHE (Personal, Social, Health and Education) lessons used for discussion about student aspirations for the new build; • the school council acting as a conduit for student voice; and
460
A. Woodcock et al.
• questionnaires sent home to encourage family discussion about the school in an attempt to involve the wider community. The study of these events and other activities within the schools have been analysed to show: 1. Participation is evident in either direct or indirect forms, using different methods to elicit requirements such as questionnaires, comment boxes, involving different groups of pupils (such as vertical groups, workshops, student councils), dealing with different design issues – fro, concept design through to the design of specific spaces, or pieces of equipment (such as window locks). 2. The organisation of consultation and participatory exercises are mostly ill structured, with little continuity observed between consecutive meetings 3. Different ways in which the ethos of the school is reflected in its architecture and interior décor 4. The way in which buildings are created and inhabited and the factors which contribute to their creation, such as • The extent of the overall vision for the school (eg whether this is just a refurbishment, the chance for the school to embrace new teaching and learning concepts, or reach out to the community) • The broader-socio economic context • The school’s identity at regional and national level • The level and(perceived) locus of expertise • The drive of the extended schools agenda and links with local communities • The effects of national and local drivers, such as preferred colour schemes, the use of natural and sustainable material.
Conclusions Our research with 11 schools and interviews with Local Education Authorities indicates that schools are not being provided with sufficient guidance or methods to enable pupil (or even teacher) participation in the design of new schools. This is problematic given that involvement in the creation of large-scale community projects can increase a feeling of ownership and co-operation, the enormous investment being placed in the building programme and the hopes this is raising for the reinvigoration of the UK educational system. Through observations of typical (as opposed to flagship or academy) schools and interviews with LEAs, we have found that the protracted nature and complexity of the design process severely limits the possibilities for pupil (and teacher) participation. The use of participation is further hampered through the lack of time and expertise in involving pupils in design on various levels, a lack of trained facilitators and absence of a good-practice guide. Although we would agree that the type of participation in schools should not be prescribed, it is still necessary to provide schools with some guidelines, and ensure that participatory discussion with pupils and teachers goes beyond a discussion of toilet and eating facilities, without
Pupil involvement in classroom (re)design
461
unnecessarily raising expectations. Interviews with staff and students suggest that children can understand budgetary and time constraints, and whilst some of their ideas may be aspirational and unrealistic, they can work with design constraints. We conclude that policy-making regarding participatory design/ergonomics should be better grounded in the complex and diverse realities of the (re)design of school environments in practice. The experiences of schools that have already embarked on the BSF process and have developed approaches to engage their students should be collated to provide a set of methods that could be developed into a toolkit as a resource for the next wave of schools that will be involved in the programme. Additionally greater attention should be given to the participation of teachers and the types of educational practice which the new schools will support.
References DCMS (2000) Better Public Buildings: a proud legacy for the future. Department of Culture Media and Sport, London. DfES/SBDU (2002) Schools for the Future: designs for learning communities. London, DfES. DfES [Department for Education and Skills] (2004a) Transforming Schools: an inspirational guide to remodelling secondary schools. London, DfES. DfES [Department for Education and Skills] (2004b) Project Process to ensure BSF aims are met. http://www.bsf.gov.uk/bsf/process.htm DfES [Department for Education and Skills] (2006) The Five year Strategy for Children and Learners: maintaining the excellent progress. London, DfES. DfES [Department for Education and Skills] (2007a) Standard Specifications, Layouts and Dimensions 1: partitions in schools. London, DfES. DfES [Department for Education and Skills] (2007b) Standard Specifications, Layouts and Dimensions 3: toilets in schools. London, DfES. DfES [Department for Education and Skills] (2007c) Standard Specifications, Layouts and Dimensions 2: floor finishes in schools. London, DfES. Horton, J. 2007, Review of literature: UK national policy context, Internal Project Document. Newman, M. and Thomas, P. 2008, Student participation in school design; one school’s approach to student engagement in the BSF process, CoDesign, 4, 4, 237–251. PriceWaterhouseCooper, 2001. Building Performance: an empirical assessment of the relationship between schools capital investment and pupil performance. Research Report 242. Department for Education and Employment, London.
RESULTS FROM A POST-OCCUPANCY EVALUATION OF FIVE PRIMARY SCHOOLS M. Newman, A. Woodcock & P. Dunham Coventry University, UK Since 1998 several primary schools have been built in Coventry that follow the guidelines contained in a model brief developed by Coventry City Council. This paper presents an overview of key findings from the implementation of a Post Occupancy Evaluation (POE) toolkit. This was the first to be specifically designed for U.K. primary schools and formed a systematic approach to evaluating the effectiveness of these buildings. The POE comprised an in-depth questionnaire for all adult stakeholders and a scheme of work designed for children. Approximately 1600 people were involved in the survey. A broad overview of the significant findings of the research will be presented, including an evaluation of key aspects of the model brief.
Introduction Coventry city council introduced its model brief for primary school builds in 1998. However, until this research was conducted no evaluation as to the effectiveness of the brief and the design principles it promotes had been conducted. This is despite the wide acceptance that the environmental conditions of schools can have a significant impact on the health and safety of children and adults in school, affecting the ways in which teachers teach and children learn. It has been affirmed that conditions in schools will be improved by effective evaluation and implementation of corrective action (DCMS, 2000, PricewaterhouseCooper, 2001, Higgins et al., 2005, Bellomo, 2006). The toolkit was developed as part of Newman’s PhD research, building on the knowledge gained from a set of earlier case studies that had examined schools built in the 1970s and 80s to assess how far they met user needs. These initial case studies concluded that many schools built at this time did not sufficiently meet the needs of the users or of the demands of current approaches to education or teaching methods (Newman et al., 2007). The purpose of the toolkit was to evaluate the extent to which the problems highlighted in the initial research have been alleviated in more recent builds. There is a growing body of evidence from the field of ergonomics indicating the physical effects that the school building may have on the child, including elements such as classroom size/density, air quality, temperature and acoustics, which have a tangible effect on learning (Hathaway, 1990, Evans, 1997, Tanner, 2000, Lundquist, 2000, Shield, 2003, Ulrich, 2004, Wyon, 2004,
462
Results from a post-occupancy evaluation of five primary schools
463
Higgins et al., 2005, Shaughnessy et al., 2006, Maxwell, 2006, Stevenson, 2006, Woolner et al., 2007). The research therefore interrogated the extent to which the new builds met the physical requirements of its users. Unlike the car industry, where performance requirements and customer feedback have long since been part of the ongoing design process, until recently POE has not been seen as a necessary part of the design and construction of new buildings (Douglas, 1994). However, there is now an increasing body of literature concerned with the necessity of post occupancy evaluation as part of an ongoing process of design. It is defined as “the process of systematically evaluating the degree to which occupied buildings meet user needs and organizational goals” (Lackney, 2001:2). A POE provides “an appraisal of the degree to which a designed setting satisfies and supports explicit and implicit human needs and values for whom a building is designed” (Friedman, 1978:20). The need for effective POE in the educational setting has been recognised (Lackney, 2001, Sanoff, 2001). It should aim to assess the extent to which the building supports the educational goals of the school by measuring its physical appropriateness to its function (Hawkins, 1998). POE should “describe, interpret and explain the performance of a school building” (Sanoff, 2001:7). However building assessment and evaluations have rarely examined the relationship between the physicality of the school and the educational goals of the establishment (Lackney, 2001). The toolkit that was implemented in order to evaluate the schools was the first to be designed specifically for use in the primary school setting. The UK government is about to embark on a major program of rebuilding and refurbishment of primary schools. Under the Primary Capital Programme half of all UK primary schools will be refurbished or rebuilt by 2022–3 with £1.9 billion spend allocated for the years 2008–11. It is therefore essential that effective evaluation is conducted as an ongoing element of the design process in order to build upon lessons learned in previous builds.
Background of schools The five primary schools that were involved in the research were all built post1998 in the city of Coventry in the midlands of the U.K. They have been given pseudonyms to maintain anonymity. Woodleigh Primary School is single form entry and was opened in August 2004 after the closure of two existing primary schools. The New Deal for Communities project was involved in the financing and building of the new school. The school is built to a triangular configuration, with the early years centre (supported by New Deals for Communities) along one side, the primary school on the second, and a large community/conference centre on the third. The building therefore accommodates three separate but connected functions. The school will also have a separate special needs school built within its site in the year 2010. Holyhead is a large primary school near the centre of Coventry, which serves a mixed population. The new school building opened in September 2006. It is a two
464
M. Newman, A. Woodcock & P. Dunham
form entry, meaning that each year groups has approximately 60 pupils. One of the striking features of the school building is its growing roof of sedum. Windbrook Primary School is a grant maintained Primary School for pupils aged 3 to 11 and was the first school to be built to the city’s model brief in 1998. The new build replaced the previous Victorian building. Most of the old school has since been demolished but one part is still used today as the Nursery and the Community Lounge. It is a single form entry, which means it is a slightly smaller than the average size school. Grafton is a community primary school with approximately 250 pupils on its roll. It is a new school created in 2003 as a result of the amalgamation of two primary schools. The school moved into its new building in January 2006. The school has an innovative design, with an oval central hall and corridor which runs around it. The school architects received several design awards as a result of their work on Grafton, however it was the most heavily criticised by teachers and children in terms of suitability for purpose. Croft Park is a large two form entry primary school, with 440 pupils on its roll, plus provision for 78 part-time nursery places. The school was rebuilt in 2001. According to the most recent OFSTED report it now has very good accommodation, which is used by a private out-of-school care provider and groups in the local community. All except Croft Park are situated in areas of high socio-economic deprivation.
Method The POE was implemented in five primary schools, all built since the adoption of the Coventry model brief. The toolkit was designed to gain a multi-stakeholder evaluation and involved the use of a set of in-depth questionnaires specifically designed for each adult stakeholder group. In order to gauge whether the new primary schools met the needs of children, two child-centred schemes of work were designed. Children aged 8–11 completed a workbook that asked them about aspects of their daily school experience and the extent to which the environment was compatible with their needs. Younger children, aged 4–7 were read a story, written by the first author, about the daily activities of school. As they listened they completed a worksheet to give an evaluation of the school buildings and outside space and whether the needs for their daily activities were met by the school. The entire populations of the five schools were surveyed, which comprised approximately 1600 people. Return rates varied with the schools and the user groups. At the time of the research only six schools had been built according to the model brief, therefore the involvement of five schools in the research represented 83% of the entire population inhabiting model brief schools. Response rates from teachers, senior teaching staff (for example head teachers and deputy head teachers) and from children was generally >80%. Other user groups had lower response rates, and the response rate from parents was generally very low, in the case of two of the schools <10%. In order to provide statistically
Results from a post-occupancy evaluation of five primary schools
465
significant results this paper will concentrate on the responses from the teachers and children. The research was conducted during 2007 and 2008.
Results The results presented will be an overview of the major issues to arise from the findings of the research. It is impossible within the constraints of this paper to give an in-depth analysis of school-specific issues therefore the results will concentrate on issues pertaining to the general principles of the model brief.
The classroom Coventry’s model design brief for primary school states that “. . . the curriculum is the focus when designing new or remodelled schools. The way in which the curriculum is organised, managed and delivered for the benefit of all pupils must be uppermost in the minds of those who have responsibility for its design. The final building must enhance the quality of the provided and received education. It should also ensure a flexible approach to teaching and learning that allows for the implementation of changing philosophies.” (2005:5) (Author’s italics) The design principles contained within the document seek to promote an organisational strategy that encourages a flexible approach to teaching and learning. The clearest way that flexibility is embedded in the physical structure of the building is through the inclusion of sliding doors between classrooms. This may be in pairs of classrooms, or in some cases, between a set of three classrooms. Teachers were asked whether sliding doors between classrooms were a good idea. 51.1% of teachers agreed or strongly agreed that having the doors were a good idea. 22.2% of teachers either disagreed or strongly disagreed. However when the children were asked how often the doors were used 44.4% of children stated that the doors were either never opened or only used on special occasions. Some of the classrooms used the doors so infrequently that many children (20.3%) stated that their classrooms did not have them. The children’s workbook which formed a large part of the survey for key stage two children asked for comments on aspects of the classrooms. A recurrent theme was the potential for distraction that having glazed sliding doors had on children. On visits to the schools, the first author noted that several classrooms had the sliding doors blocked by the positioning of bookcases or other furniture, rendering it extremely difficult to open the doors regularly. Teachers commented that their use depended on a team approach that was not consistently adopted by all teaching staff. Clearly there is a disparity between the type of flexible approach to teaching and learning promoted by the local authority, and the methods and approaches to teaching used at the classroom level.
466
M. Newman, A. Woodcock & P. Dunham
Another key element of the design brief in the promotion of flexibility was the inclusion of an area outside the classroom space which was shared between two or more classrooms. Responses to an open question on how this space is used indicate that it is utilised in a variety of ways by the different schools. For example as a withdrawal area where children who have specific needs go to work in small groups, an art and craft area or to house additional computers to allow children ready access to ICT.Teachers were asked whether having a shared area outside the classroom helped teaching and learning. 75.6% of teachers gave a positive response to this question. 80.5% of children thought that having the space away from the classroom was a good idea. The model design brief, as used in the construction of the subject schools, utilised the guidelines outlined in Building Bulletin 82 (DFES, 1996), which have since been superseded by Building Bulletin 99 (DFES, 2004). The incorporation of a shared area into the design brief meant that some of the floor space that could have expanded the main body of the classroom was used for the shared area. However, the results from the survey indicated that this was not a major problem for either staff or children, with 80.9% of children and 66.7% of teachers agreeing that the classroom was big enough. There was no indication that using some of the allocated floor space for a shared area raised the classroom density to an unacceptable level. Teachers generally thought that the design of the classroom facilitated learning, with 88.4% providing a positive response to the question. A problem that appeared consistently in all schools was that of temperature control and the impact that this had on the classroom temperature. The connection between thermal discomfort, low air quality and poor classroom performance has been recognised for several decades (Lofstedt, 1969, Griffitt, 1971, Jaakkola, 1989, Lee and Chang, 1999, Wargocki et al., 2005, Jaakkola, 2006). The lack of control of temperature was a significant issue for both teachers and children. 65.1% of teachers stated that they had no control over temperature in their classroom and that it was frequently uncomfortably hot. 73.7% of children stated that their classrooms often got too hot. This was a consistent response, although the schools were involved in the survey throughout the school year, not solely in the summer months when a higher temperature may be expected. Open questions on what teachers would like to change about the classroom environment indicated that the ability to control temperature within their classroom rather than at a central point would increase thermal comfort and would be desirable. Teachers from Grafton School stated that the under-floor heating was a particular problem, especially when children were asked to sit on the floor for whole class activities, for example listening to stories. Acoustics were generally considered good with both teachers and children agreeing that audibility of the teacher’s speech was clear. 75% of teachers felt that acoustics were good in the classroom, whilst 97.2% of children stated that they could hear the teacher clearly in the classroom.
Dining room/hall Dining rooms were generally considered pleasant places to eat, both by children and by the kitchen staff and lunchtime supervisors, with enough space to walk between tables, the food on offer was clearly visible, there was enough space for children to
Results from a post-occupancy evaluation of five primary schools
467
sit with friends and there were not great distances to negotiate between the servery and tables.
Outside environment The design of the outside environment of schools is considered of great significance to primary education (Malone, 2003, Malone and Tranter, 2003, Tranter, 2004, Armitage, 2005). Playgrounds and the outdoor classrooms for foundation stage children were generally considered well designed, with access to natural habitats and activities to encourage imaginative play. Quiet areas were also provided along with shaded areas to ensure that children had the option to sit or play quietly in a sheltered area.
Other areas Several other areas of the school were evaluated, including the provision of small group rooms, libraries and ICT. Responses were generally positive about the provision of these facilities, which were felt to be adequate. A separate library and ICT suite was preferred by teachers and children to the inclusion of these facilities into corridors. From the open questions in the scheme of work it was clear that a significant proportion of children felt although the provision within ICT suites were good, they would like to have more ready access to ICT within their classroom to include the use of computers within literacy and other lessons.
Conclusions The general picture gained from the research was that the newly built primary schools were effective environments for teaching and learning. The new schools were evaluated to be an improvement on schools built prior to the implementation of the model brief. They provided good physical environments and classrooms were generally perceived as being conducive to teaching and learning, supporting current educational philosophy and the requirements of teachers and children as they go about their daily activities. The shared area outside the classroom has been evaluated as being a particularly efficient use of space, having many and frequent uses, gaining the approval of the majority of key user groups. Other areas in the school, such as dining room, quiet rooms and the outdoor classroom and playgrounds were also considered to be adequate. Certain aspects of the model brief and of new school design however need to be reassessed in the light of these findings. The inclusion of sliding doors is often superfluous to the teaching methods and approaches adopted by those involved in the everyday teaching practices at the schools. It may be that rather than excluding the doors from the design of future schools, it is necessary to provide further training for teachers to encourage the implementation of a mode of collaborative teaching.
468
M. Newman, A. Woodcock & P. Dunham
The control of temperature in classrooms was a major issue, particularly in schools where under-floor heating was used. The research indicates that more traditional means of heating classrooms, although less aesthetically pleasing, may be more appropriate. Temperature should also be controllable within the classroom, rather than through a central control. Post-occupancy evaluation should be a key part of the design process to ensure the improvement of design and that the needs of all stakeholders are met. The use of the specifically designed primary school post-occupancy toolkit has evaluated the model brief as used in Coventry providing a valuable insight into the effectiveness of a set of design principles that had hitherto been untested. This included a means of gauging the usability of the environment for all users, including young children. Its implementation has also provided a method that may be developed further for use in other UK education authorities to enable effective evaluation in order to build upon successes and avoid problems associated with earlier building programmes. This is particularly timely as the UK government embarks upon the Primary Capital programme which aims to place primary schools as a central part of the community.
References Armitage, M. (2005) The Influence of School Architecture and Design on the Outdoor Play Experience within the Primary School. Paedagogica Historica, 41, 535–553. Bellomo, A. J. (2006) Evaluating the school environment. IN FRUMKIN, H., GELLER, R., RUBIN, I. L., NODVIN, J. (Ed.) Safe and Healthy School Environments. New York, Oxford University Press. Coventry (2005) Building Primary Schools for the Future: Model Brief and Design Brief for New and Remodelled Primary Schools 2001. Coventry, Coventry City Council, Education and Library Service. DCMS (2000) Better Public Buildings: a proud legacy for the future. Department of Culture Media and Sport, London. DFES (1996) Building Bulletin 82: Area Guidelines for Schools. HMSO. DFES (2004) Briefing Framework for Primary School Projects, Building Bulletin 99. London: The Stationery Office. Douglas, J. (1994) Developments in appraising the total performance of buildings. Structural Survey, 12, 10–15. Evans, G. W., and Maxwell, L. E. (1997) Chronic noise exposure and reading deficits : The mediating effects of language acquisition. Environment and Behaviour, 29, 638–656. Friedman, A., Zimring, C., and Zube, E. (1978) Environmental Design Evaluation, New York, Plenum. Griffitt, W., and Veitch, R. (1971) Hot and crowded: Influences of population densities and temperature on interpersonal affective behaviour. Journal of Personality and Social Psychology, 17, 92–98. Hathaway, W. E. (1990) A Study into the Effects of Light on Children – A Case of Daylight Robbery. IRC internal report.
Results from a post-occupancy evaluation of five primary schools
469
Hawkins, H. L., and Lilley, H. E. (1998) Guide for school facility appraisal. Scottsdale, AZ, USA, Council for Educational Facility Planners, International. Higgins, S., Hall, E., Wall, K., Woolner, P. & McCaughey, C. (2005) The Impact of School Environments: A Literature Review. University of Newcastle. Jaakkola, J. J. K. (2006) Temperature and Humidity. IN FRUMKIN, H., GELLER, R., RUBIN, I. L., NODVIN, J. (Ed.) Safe and healthy School Environments. New York, Oxford University Press. Jaakkola, J. J. K., Jaakola, N., and Sappenen, O. (1989) Sick building syndrome, sensation of dryness and thermal comfort in relation to room temperature in an office building: Need for individual control of temperature. Environ Int, 15, 163–168. Lackney, J. (2001) The state of post-occupancy evaluation in the practice of educational design. 32nd Annual Meeting of the Environmental Research Association. Edinburgh, Scotland. Lee, S. C. & Chang, M. (1999) IndoorAir Quality Investigations at Five Classrooms. Indoor Air, 9, 134. Lofstedt, B., Ryd, H., and Wyon, D. (1969) How classroom temperatures affect performance on school work. Build International, 2, 23–34. Lundquist, P., Holmberg, K., and Landstrom, U. (2000) Annoyance and effects on work from environmental noise at school. Noise and Health, 28, 39–46. Malone, K., and Tranter, P. (2003) School Grounds as Sites for Learning: making the most of environmental opportunities. Environmental Education Research, 9, 283–303. Malone, K. & Tranter, P. (2003) Children’s environmental learning and the use, design and management of schoolgrounds. Children, Youth and Environments, 13. Maxwell, L. (2006) Crowding, Class Size and School Size. IN FRUMKIN, H., GELLER, R., RUBIN, I. L., NODVIN, J. (Ed.) Safe and Healthy School Environments. New York, Oxford University Press. Newman, M., Woodcock, A. & Dunham, P. (2007) How children perceive and use the primary school environment. IN BUST, P. D. (Ed. Contemporary Ergonomics. Loughborough University, Taylor and Francis. PricewaterhouseCooper (2001) Building Performance: an empirical assessment of the relationship between schools capital investment and pupil performance. Research Report 242. Department for Education and Employment, London. Sanoff, H., Pasalar, C., and Hashas, M. (2001) School Building Assessment Methods. National Clearing House for Educational Facilities www.edfacilities. org/pubs/sanoffassess.pdf Shaughnessy, R. J., Haverinen-Shaughnessy, U., Nevalainen, A. & Moschandreas, D. (2006) A preliminary study on the association between ventilation rates in classrooms and student performance. Indoor Air, 16, 465–468. Shield, B. M., and Dockrell, J. E. (2003) The effects of noise on children at school: a review. Journal of Building Acoustics, 10, 97–106. Stevenson, K. R. (2006) School size and its relationship to student outcomes and school climate. National Clearing House for Educational Facilities.
470
M. Newman, A. Woodcock & P. Dunham
Tanner, K. (2000) The influence of school architecture on academic achievement. Journal of Educational Administration, 38, 309–330. Tranter, P., Malone, K (2004) Geographies of environmental learning: an exploration of children’s use of school grounds. Children’s Geographies, 2, 131–155. Ulrich, C. (2004) Children and the Physical Environment. Human Ecology, 32, 11–14. Wargocki, P., Wyon, D. P., Matysiak, B. & Irgens, S. (2005) The effects of classroom air temperature and outdoor air supply rate on the performance of school work by children. Indoor Air ’05. Woolner, P., Hall, E., Higgins, S., McCaughey, C. & Wall, K. (2007) A sound foundation? What we know about the impact of environments on learning and the implications for Building Schools for the Future. Oxford Review of Education, 33, 47–70. Wyon, D. P. (2004) The effects of indoor air quality on performance and productivity. Indoor Air, 14, 92–101.
STRATEGIC ERGONOMICS
REPOSITIONING HUMAN FACTORS/ERGONOMICS (HF/E) – RAISING HIGH HAZARD INDUSTRY UNDERSTANDING AND COMMITMENT TO HF/E D. Pennie1 , M. Wright1 & P. Mullins2 2
1 Greenstreet Berman Health and Safety Executive
Health and Safety Executive (HSE) considers itself a leader and opinion former and seeks to promote Human Factors Ergonomics (HF/E) in a number of ways using different types of media (internet, seminars etc). HSE are aware, however, that, although successful in communicating with safety specialists, knowledge about HF/E does not always penetrate through to the Key Decision Makers (KDMs). The research involved interviews with 60 KDMs from a range of high hazard industries. The objectives of the interviews are to understand how decision makers address process safety; why the message concerning HF/E does not always ‘get-through’ and how HSE can secure greater commitment to the application of HF/E. Findings suggest that there maybe a number of different strategies for improving knowledge and application of HF/E in tackling process safety. For example: deliver a consistent message on HF/E; continue to help develop HF/E knowledge and encourage peer evaluation and sharing of good practice.
Introduction Health and Safety Executive (HSE) considers it self a leader and opinion former. For more than twenty years they have promoted the adoption of Human Factors/Ergonomics (HF/E) in a number of different ways and through using different types of media (internet, seminars etc). The HSE also provide: • A suite of guidance material on, for example, human error; manual handling and shiftwork; • An “operationalised” list of key HF/E topics with briefing sheets and extracts from an inspectors toolkit and other useful resources; • Practical tools (Fatigue Risk Index and the Manual Handling Assessment Chart). HSE have also recruited specialist HF/E inspectors, and provide additional training on HF/E for Health & Safety (H&S) field inspectors. The role of these inspectors is further supported by a dedicated HF/E unit. In addition formalised HF/E guidance is provided, to be used in conjunction with The Control of Major Accident 473
474
D. Pennie, M. Wright & P. Mullins
Hazards Regulations (COMAH) 1999 and Nuclear Safety Assessment Principles (SAP). Over the last 5–10 years it is generally recognised that these measures have been effective in engaging with the industry and increasing understanding of this topic, particularly among safety professionals. Concerns have been raised, however, that HF/E does not always penetrate through to KDMs (such as site and operations managers, engineers etc) with the consequence that HF/E is not always integrated into day to day decisions. Earlier research (Wright et al., 2003) indicated that this may in part be due to a hazy perception of HF/E. This inhibits understanding of HF/E related issues and how to deal with them. It also found that HF/E is: • Viewed as being ill defined and nebulous, without clear boundaries; • Equated by some managers to behavioural safety (changing attitudes and influencing behaviour towards safety); • Seen as a nice to have but not seen as essential; • Not fully integrated into H&S management. The research concluded that HSE should continue to promote HF/E by increasing awareness and understanding. It should seek to gain greater commitment to the management of human error, and in particular greater application of the principles and methods that encompass HF/E. The HSE has also identified a requirement to: widen the breadth of people exposed to HF/E knowledge and overcome the barriers that might be affecting this broader uptake. This paper reports on ongoing work currently being conducted in 2008, by HSE, to explore this issue further and build upon this previous research. This work is intended to assist HSE with the overall objective of improving the management of major hazards through increasing the uptake of HF/E. Its aim is to: • Identify key decision makers; • Explore how HF/E is currently applied to address issues concerning process safety; • Outline the barriers to the uptake of HF/E; • Discuss how HSE might promote the greater application of HF/E.
Method The work is qualitative research using interviews with a cross section of representatives from high hazard industries within the following sectors: • Nuclear; • Chemicals; • Oil and Gas (offshore & onshore). Initially 15 interviews were conducted with a range of stakeholders who were identified as having knowledge and experience of tackling HF/E and/or safety. Interviewees were asked to consider: who are the KDMs; how decisions are made and what methods are used to support these decisions (including the incorporation of HF/E principles into the decision making process).
Repositioning human factors/ergonomics (HF/E)
Table 1.
475
Main interview profile.
Key Decision Maker (KDM)
Chemicals
Oil and Gas (Onshore)
Oil and Gas (Offshore)
Nuclear
Total
Senior Manager Site Managers Front Line manager H&S Personnel Total
3 6 5 1 15
2 5 6 6 19
4 2 3 5 14
3 3 5 1 12
12 16 19 13 60
The findings from these interviews were used to generate a decision making profile and then this profile was used to recruit a further 60 interviewees, to participate in the main part of the research – see Table 1. These interviewees were asked to discuss: • Experience of involvement in a decision, development activity or assessment process that directly or indirectly influenced process safety; • What ways they had learnt more about process safety and ways of improving it; • The use of applied approaches, principles or techniques to assess error, manage human performance or influence behaviour; • Options for further enhancing application of HF/E to support process safety decision making. It should be noted that interviewees self selected to take part in the interviews. This means it is likely that respondents have a more positive attitude towards both HSE and the application of HF/E.
Findings The findings are presented under the four key topics of: • • • •
Identifying Key Decision Makers (KDMs) in the high hazard industries; Current application of HF/E; Barriers and lessons learnt; Future options for improving the uptake and application of HF/E.
Identifying the key decision makers and exploring how HF/E decisions are made The findings from the initial interviews, with 15 stakeholders, largely confirmed current thinking on decision making. There were four main role types, most cited by interviewees, as making decisions that take account of HF/E issues (influencing human error, performance and behaviour) namely: • Senior management (site director, managing director, senior executive); • Site Managers (facility, installation or refinery managers – those leading dayto-day plant operation);
476
D. Pennie, M. Wright & P. Mullins
• Front line managers (operations managers, production managers, project managers, design managers, safety assurance managers, maintenance managers); • Health and safety personnel (HSE department, safety director, safety manager, safety case authors, incident investigators, process safety specialists etc); These four main role types were the same for each of the different sectors of: Nuclear, Chemicals and Onshore/Offshore Oil and Gas. Interviewee responses indicated that a very diverse range of people and departments were involved in making decisions. The findings also indicated that decisions, about HF/E, also occur throughout the hierarchy of the organisation. For example, individuals at a very senior level are making decisions on topics like organisational change and culture (which can have an impact company wide) yet HF/E is also relevant to those making decisions concerning day-to-day operation, such as, alarm handling and procedures. This provides evidence that HF/E is a wide ranging topic relevant to many aspects of managing and operating a company. It also highlights a challenge of communicating about HF/E because it must reach a different and varied audience. Decisions that take account of HF/E issues were often made collaboratively and in teams, for example: Refinery Management Team; SHE team; Personal Safety Team; Safety and Operations Team; Process Safety Team. The findings indicate that decisions on HF/E are made in different ways. The most frequently cited method was a top down approach with decision making driven by company policy; corporate standards or programmes or initiatives. Decisions to tackle a particular issue were also influenced by incidents and accidents both internal and external to the organization as well as a regular review process. The other key reason for tackling a particular issue or using a HF/E approach was through regulation and visits from HSE inspectors (generally as part of their normal duties to support top tier COMAH sites). “The key driver was a HSE intervention following an accident related to insufficient labelling of plant equipment – The organization was told to include a HF view/approach.”
Current application of HF/E Interviewees were asked to discuss recent examples where they were involved in decisions that influenced process safety. Many of the examples concerned plant modifications and improvements to system design but there were also a wide range of the typical and familiar HF/E issues considered. For example: emergency shut down; alarm handling; manning levels; training & competency; shiftwork schedules; procedures; leadership; safety culture; controls; management of change: maintenance; communication. This is similar to the operationalised list of key HF/E topic areas, outlined on the HSE major accident hazard website. This would either indicate that HSE are providing industry with the right type of information, or that industry is following the lead of HSE. Decision making on process safety was supported by the use of generic techniques such as risk assessment and HAZOP studies. There was also a well developed understanding of the role of human error in incident causation and a number
Repositioning human factors/ergonomics (HF/E)
477
of interviewees cited the use of tools, such as, root cause analysis and recent improvements in the way incident investigations are conducted. There was recognition that in the past there may have been a greater tendency to focus on the role of the individual in causation. “In the past there has tended for blame to be placed on the individual but now considerations is given into how that individual was put into a position where the incident could occur” A number of interviewees commented that their organisation was now better at identifying the deeper underlying causes of an incident, and a number of techniques were described that helped to do this. “We have integrated human factors into abnormal incident methodology – using root cause analysis. As part of this, guidance of analysing human factors has also been re-developed”. In addition, behavioural modification methods were frequently cited by decision makers in the oil and gas sector. In the nuclear industry there also appeared to be greater use of more specific analytical tools, such as: training needs analysis; task analysis and gap analysis. The nuclear sector also had more developed in-house HF/E support provided by human performance specialists. Overall, however, the findings from the interviews indicated that such tools and techniques that support decision making on HF/E issues were considered to be outside of their expertise and should be the preserve of HF/E specialists. “Personally I find a little bit of knowledge a dangerous thing. You need the experience and background to apply these techniques and to use them regularly. There is a danger of applying your own interpretation to suit your view. It needs autonomy and should be left to the experts”. KDMs were also asked to rate their awareness; understanding and attitude towards HF/E. The findings indicate that the 88% of KDMs believed it is relevant to accident prevention and 91% thought it was important for their business. KDMs indicated that they had good awareness of HF/E (64%) but only 36% stated they had a good or very good understanding of the subject. KDMs also gave a number of reasons that had prompted them or their organisation into giving greater consideration of HF/E issues, for example: • Visits by external evaluators (government inspectorate; commissioning agents etc); • Reports investigating high profile accidents within their sector or where the incident has relevance to their own operation (Texas City, Buncefield); • In house processes that had identified a problem in a specific area (incident and event analysis; audits; workforce consultation etc); • Networking and partnerships - exchanging knowledge with peers within the same industry/sector (in the nuclear sector this is facilitated through the World Association of Nuclear Operators WANO); • Attending industry association events (meetings, conferences). Knowledge of process safety, and ways of improving process safety through consideration of HF/E issues, were largely developed in-house and through on-the-job experience. Some interviewees had also stated that they had developed their knowledge through their degree, or by attending external training courses, however, this route tended to be less common. Interviewee responses indicated that additional
478
D. Pennie, M. Wright & P. Mullins
knowledge was more likely to be gained through networking and partnerships as well as studying the outcomes from major accident investigations.
Barriers and lessons learnt The study identified a number of key barriers which may affect the penetration of HF/E knowledge and techniques. It also provided some very useful insight into the preferences of KDMs and what might help to encourage the wider uptake of HF/E techniques and principles. A significant barrier was inconsistency. Interviewees cited a number of examples where inconsistency was a problem in both their understanding and application of HF/E. For example, one interviewee stated that outside HF/E specialists, that had been brought in to assist them tackle a particular issue, used different and contrary techniques to those advocated by HSE. Inconsistency also extended to the experience of HSE inspectors who might themselves have different levels of understanding of HF/E. “Need to bring local inspectors up to speed on HF and give industry more access to HSE HF specialists”. The inconsistency that decision makers comment upon is indicative of a problem that exists within the discipline of HF/E. There are many techniques and applications originating from different fields of research; industry and within different sectors. Tools and techniques are also often adapted or altered to a new area. This is illustrated by the many different techniques for determining error probabilities. The use of guidance and other information provided, for example, on the HSE website was very seldom cited as a method for gaining a greater understanding of HF/E. Overall, however, the general view was that support and advice provided from HSE was good particularly from visiting inspectors. A commonly expressed view, however, was that inspectors were good at identifying where an organisation was less well developed, in terms of HF/E, but did not provide sufficient practical guidance on how that organisation might actually improve. This may indicate that there are simply not the resources within the HSE to provide the desired level of support. Internal resources were also cited as a barrier to greater uptake of HF/E. “Really need buy-in from senior management so that resources can be directed to tackle human error”. One interviewee also stated that it was sometimes difficult to resist the conflicting commercial pressures that might adversely influence day-to-day operation and this was a challenge to greater uptake of HF/E. In addition to identifying some of the barriers the research has also provided insight into the more effective ways that industry has developed its understanding of HF/E. The strongest message emerging from the interviews was that the preferred way to gain a great insight in the benefits that HF/E was through sharing experiences and knowledge with peers, particularly through partnership working. The nuclear industry used ‘peer review programmes’ facilitated by the World Association of Nuclear Operators (WANO). This involved visits from technical experts who spend several weeks immersing themselves in a specific topic. At the end of the review they
Repositioning human factors/ergonomics (HF/E)
479
discuss issues they have identified and present the evidence for their conclusions. This was considered to be a highly effective and particularly compelling way of communicating on HF/E. This was further supported by the sharing of good practice with peer reviewers providing help and their own experiences of addressing the identified issues. Interviewees in the nuclear sector believed that their industry was more open to sharing knowledge, because there was a vested interest in co-operation. If a serious accident occurred then the whole industry suffered. This knowledge sharing approach, however, was also cited by other sectors although it was conducted in a less formalised way and instead developed through ad hoc contacts within their industry. Benchmarking on how human performance issues are tackled by other organisations; facilities or sites, within the same sector, was another commonly cited method to identify areas for improvement. The other method widely cited by interviewees for gaining greater insight into HF/E and process safety was through learning from past accidents, for example: Texas City, Buncefield and Longford. One interviewee stated that they had formalised learning from accidents using a procedure called: “Offsite Incident Process”. This explores the incident by holding an investigation as if the accident had occurred on their site. The findings from the interview also stated that practical examples of how an issue or problem had been tackled, was also a very effective way of communicating about HF/E. It was more important to show what was done and not simply focus on the business benefits. For example, providing an example of using a particular HF/E tool; how it provided insight an issue and how it helped to address it. The examples should also not just focus on safety issues but should also show how operational and engineering problems were tackled. Using practical examples to improve understanding of process safety was also considered a very effective way of learning about HF/E issues. For example, conducting a walkthrough of a shutdown exercise could highlight how the location of a valve or indicator panel might increase opportunity for human error.
Options for improving the uptake and application of HF/E The research also asked key decision makers to consider and rate seven different options for improving the uptake of HF/E knowledge and application. These covered areas, such as: improving the definition of HF/E; more enforcement; better integration; increasing HF/E knowledge; better communication; more initiatives; HF/E led by industry associations. The following table ranks these options with 1 being the most preferred option. In the short term it was believed to be important to continue to develop knowledge of HF/E through training and practical examples. However, they need to be relevant and engaging. A number of interviewees also believed it was important that training was accredited to ensure quality and consistency. “Courses tend to feel very remote and separate. They don’t directly apply to the rest of the work done in industry.” “Important to build understanding through education but with formal qualifications”.
480
D. Pennie, M. Wright & P. Mullins
Table 2.
Options for improving the application of HF/E.
Future options
%
rank
Improve knowledge of Human Factors through the provision of training and professional development programmes Integrate Human Factors to build knowledge, techniques and requirements IMPLICITLY into standards A programme of activity led by industry associations and professional institutes, with HSE support HSE to develop a communications campaign to raise awareness and understanding of Human Factors HSE to better explain the concept of Human Factors and provide a clearer definition of what it is. The provision of more branded initiatives, such as Step Change or Hearts and Minds, HSE to increase enforcement of Human Factors as part of safety case and related inspection processes.
74
1
67
2
64
3
54
4
54
5
43
6
33
7
In the long term, the key strategy was seen as the greater integration of HF/E with other disciplines such that it becomes embedded within, for example: engineering standards, design guidance; operational procedures etc. The view is that HF/E is too commonly seen as another box to tick and a “bolt-on topic” to current process safety activities. It needs to become part of day-to-day operations and implicit within all management thinking. This was likely to happen over time as the use of HF/E became more developed and widespread within industry, however, HSE might consider ways to help make this happen more quickly. A number of interviewees also stated that there was a danger if HF/E was integrated successfully it might dilute the message. This could be an issue, for some organizations, if the principles of HF/E were not fully accepted. “This would dilute the message that HF is critical – important to keep the message clear”. A consistent message on HF/E communicated through a more joined up campaign was seen as being beneficial. However, getting it right was seen as a challenge. Branded initiatives were also viewed positively as a way of simplifying the message about HF/E, if again done well and if industry specific. They were viewed as useful where the level of HF/E understanding was less mature but could be confusing for organisations already tackling similar issues. There is also a danger of “initiative overload” and too many branded initiatives can actually negatively affect attitudes towards HF/E. There was also divided opinion regarding the benefits of activities led by industry and supported by HSE. One view was that because industry associations have a greater knowledge of industry, they are better placed to lead on how to communicate on HF/E, “if done properly this is the best way, to work with industry”. The other view was that because the topic is complex already, it requires one institution, such as the HSE, to lead and for industry associations to follow this lead. Without a consistent message there is more likelihood for confusion in already
Repositioning human factors/ergonomics (HF/E)
481
complex topic. “In principle, a good idea, but in practice, there is less agreement between associations as to what is important.” The message about HF/E would be stronger if it came directly from HSE “Direct involvement is more effective. If it is seen as being important to HSE, as regulator, it will be important to us. In certain circumstances it was felt that enforcement could be a useful tool but overall it was believed that there was not sufficient clarity of what organisations were required to do or sufficient awareness of the types of regulation that exists on HF/E. “Everyone has heard of HSG65, but where is the standard on Human Factors? Need the standard before you can regulate”. It was also believed that it was better to work together with industry and that enforcement would not win over hearts and minds but could alienate some organizations and create mistrust. “Might just be doing it because we have to, rather than because we believe in it. Enforcement will cause people to hide things and the reporting culture may not be as good”. Not surprisingly there was no stand-out single future option for improving the application of HF/E and different approaches will have a different impact depending on how well an organisation is already applying HF/E techniques and knowledge. The research does, however, provide some insight into the current application of HF/E and the barriers to greater uptake and provides some suggestions to broaden its use. These future options will be explored at a later date with stakeholders, as part of this ongoing research.
Conclusion The research would suggest that there is good awareness of the benefits that HF/E can bring and a desire for industry to apply the principles and techniques to better understand and tackle human error and behavioural issues that can impact on process safety. A common view shared by many of the KDMs and as stated by one interviewee was: “That it is in industry’s interest to do this and everyone is working towards the same goal: a safer and more efficient workplace.” There is a desire to be better but perhaps there is still a tendency to react to events or visits from the HSE inspectorate. This might indicate that industry is still unsure of how to address the HF/E issue impacting on their business. The recent improvements in incident investigation, developing more ‘just’ reporting cultures and free learning from other events, should also help to provide greater knowledge and understanding to industry and encourage them to be more proactive. Overall HSE was seen to be communicating well particularly through the direct engagement of HSE inspectors. There is also an improved response to the need for training on human error and behaviour. Industry associations are increasing their provision of HF/E training, with courses offered by the Institute of Chemical Engineers. Other industry bodies, such as the Energy Institute, are also developing their own professional development programmes. A challenge, however, is to ensure there is a consistent message and HSE was seen as the best placed to do this. Without clear direction from HSE and allowing industry associations to lead this could create additional confusion in
482
D. Pennie, M. Wright & P. Mullins
already: “muddied waters”. A starting point might be to simply agree terminology. For example: encouraging industry and associations to consistently use the term HF/E, rather than human factors, human element or human performance. One of the most effective way of increasing knowledge and understanding of HF/E is through partnership working. A particularly compelling method was through inviting peers, with recognised expertise in a particular field, to look in detail at an organisations systems and processes by visiting their site directly. This may be a challenge for some organisations and sectors but HSE could do more to facilitate this either directly or through industry associations. HSE might also advance the breadth and depth of HF/E understanding by providing practical examples. These might show the way in which operational and engineering problems were solved through the application of HF/E knowledge and techniques. Examples, like high profile accident reporting, could also help to illustrate what can happen if there is an inadequate appreciation of HF/E issues. It is likely that, over time, HF/E will become more embedded in day-to-day plant operations with greater integration into engineering and design standards. This, however, should be achieved without losing the core message of HF/E and ensuring that its importance and profile remains strong.
References Michael Wright, Mark Bendig, Carol Hopkins, Bill Gall. 2003, The promotion of human factors in the onshore and offshore hazardous industries, RR149, (HSE Books). Geoff Simpson, Clive Tunley and Melanie Burton. 2003, Development of human factors methods and associated standards for major hazard industries, RR081, (HSE Books). Vectra Group Ltd. 2004, Human factors guidance for selecting appropriate maintenance strategies for safety in the offshore oil and gas industry, RR213, (HSE Books). White Queen Safety Strategies & Environmental Resources Management. 2007, Development of a working model of how human factors, safety management systems and wider organisational issues fit together, RR543, (HSE Books). Nickleby HFE Ltd. 2004, Human factors assessment model validation study, RR194, (HSE Books). HSE, 1999, Reducing error and influencing behaviour, HSG48, (HSE Books). HSE. 1997, Successful health and safety management, HSG 65, (HSE Books).
ERGONOMICS FOR ENGINEERS P.J. Clarkson University of Cambridge, UK Design teaching is central to the education of engineers taking accredited courses in the UK, ranging from the sketching of a concept to the detailing of an individual component, or from the identification of user needs to the analysis of a product’s performance. Hence it is interesting to reflect upon the prominence of ergonomics in a typical multidisciplinary engineering department, teaching design across this wide spectrum of disciplines and skills. At Cambridge students are introduced to design through a variety of lectures and projects, beginning within days of their arrival and culminating with the delivery of their final-year projects. This paper reflects the various elements of this course, providing the basis for a discussion of the role of ergonomics teaching within a typical engineering department.
Engineering at Cambridge “Engineering is about solving problems: about designing processes and making products to improve the quality of human life. From reservoirs to robots, aircraft to artificial hips, microchips to mobile phones – engineers design and manufacture a huge variety of objects that can make a real difference to both individuals and to societies.” This is how the Cambridge course is advertised to prospective students (Cambridge, 2008). Its aim is to provide students with all the analytical, design and computing skills that underpin modern engineering practice, while encouraging the creativity and problem-solving skills that are so important for a professional engineer. Design-related activities represent a significant educational thread running through the four-year undergraduate course (Wallace, 1993), occupying about 20% of the timetable hours in the first two years. The first year design courses include Drawing, where the students learn to develop the 3D visualisation and communication skills needed for engineering design; Product Design, where they receive an introduction to product evolution and the methods used in the first two phases of the design process (Clarification of the Task and Conceptual Design); and Structural Design, where they learn to apply the fundamentals taught in the Structural Mechanics and Materials courses to a design project, working in a team to plan, design, manufacture and test a simple structure. In addition, students attend lectures given by industrial speakers on a wide range of relevant engineering applications. The second year design courses include an Integrated Design Project, where students work in multidisciplinary teams to apply the fundamentals taught in the 483
484
P.J. Clarkson
Mechanics, Structural Mechanics, Materials, Electronics and Computing courses to a design problem, building on the skills acquired in the 1st year and developing system design and design management skills. Students also study two Electives from a choice of seven. The aim of these courses is to provide case studies and exercises associated with a much larger project than the students could tackle themselves in the time available. The engineering applications lectures continue. In the third year students undertake two 80-hour projects in the four weeks after their final examinations. Projects are offered under the following four headings: Design Projects, Computer-Based Projects, Fieldwork Projects and Foreign Language Projects. The majority of students select projects from the first two groups. The aim of these is to provide students with the opportunity to work in more depth on the Embodiment Phase of the design process and to use advanced computer-based tools to analyse and optimise given designs. In the fourth year about half of a student’s time is spent on a major open-ended project, often undertaken in collaboration with industry. The aim of the Major Project is to cover the complete design process for a challenging product or system, or to contribute to fundamental research in Engineering Science, potentially developing a novel method of design or analysis. The other half of the final year is devoted to taught modules. Of the 70 or so modules available many have a design focus specific to a particular engineering domain. In addition, there are three modules that focus on aspects of the design process itself, all of which are heavily influenced by current research in the Cambridge Engineering Design Centre.
Training engineers The structure described in the previous section is typical of many of the Engineering Masters courses offered in the UK and is in accordance with the latest guidance offered by the engineering institutions, namely UK-SPEC (Engineering Council, 2005), which describes the standard for the recognition of professional engineers and professional engineering technicians in the UK. This in turn builds upon SARTOR (Standards and Routes to Registration) which was established in the early eighties (Engineering Council, 1997) and has had far-reaching impact on engineering education (Hamilton, 2000). Both these initiatives can be traced further back to the Finniston Report (Finniston, 1980) on the Committee of Inquiry into the Engineering Profession which proposed the establishment of MEng courses as a major element of training for profession engineers. Common to all accredited courses is the need to structure teaching to contribute to specific learning outcomes, namely: underpinning science and mathematics, and associated engineering disciplines, as defined by the relevant engineering institution; engineering analysis; design; economic, social, and environmental context; and engineering practice. The particular balance chosen for these outcomes is dependent on the broader aims defined for each course (Engineering Council, 2005). The outcomes expected for the design theme are further refined, where design is defined as “the creation and development of an economically viable
Ergonomics for engineers
485
product, process or system to meet a defined need” (Engineering Council, 2005), requiring the knowledge and skills to: • Investigate and define a problem and identify constraints, including: environmental and sustainability limitations; health and safety; and risk assessment issues; • Understand customer and user needs and the importance of considerations such as aesthetics; • Identify and manage cost drivers; • Use creativity to establish innovative solutions; • Ensure fitness for purpose for all aspects of the problem, including: production; operation; maintenance and disposal; • Manage the design process and evaluate outcomes. Curiously, these descriptions say little to encourage the teaching of ergonomics. Whilst reference to “customer and user needs” and “fitness for purpose” is made, there is no explicit reference to ergonomics, i.e. the application of scientific information concerning humans to the design of products and systems for human use. In fact, a brief survey of MEng courses offered in the UK, as evidenced by their webbased course descriptions, suggest that ergonomics does not feature prominently in UK engineering education.
Design teaching at Cambridge An outline of the elements of design-related teaching has already been given. However, in order to understand the changes introduced over the past few years, to accommodate a shift in emphasis from teaching engineering design to teaching product design, it is necessary to describe two of the elements in more detail: the Product Design course presented to the whole of the first year and the Design Case-Studies course offered to final year students. These courses are strategically important since the former provides an opportunity to reach all undergraduate students early in their Cambridge careers and the latter provides a focus for those choosing to specialise, at least in part, in design.
First year product design course The Product Design course aims to: instil an appreciation of the nature and context of engineering design; emphasise the importance of designing products that meet their performance requirements safely, reliably, economically, sustainable, ergonomically and aesthetically; and introduce the basic ideas of systematic product design. At the end of the course students should be able to: prepare a problem statement for a simple engineering product; elaborate a requirements list (design specification); establish a function structure; generate and select suitable solution principles to produce concepts; evaluate concepts; and present the final concept in the form of sketches and drawings. The focus here is to introduce students to the ‘engineering’ design process.
486
P.J. Clarkson
Again, there is no direct reference to ergonomics, however, there is opportunity to introduce some principles of user-centred design. Following a series of eight lectures the students undertake a conceptual design project; where topics over the years (in chronological order) have included: fencepost extraction; rope tensioning; branch pruning; emergency braking; gardening from a wheelchair; external house painting; a car roof transporter for two bicycles; river rubbish clearance; medication packaging; making shopping easier; designing against crime; and designing for independent living. During this time the lectures presented have also changed (Table 1), with a shift away from case-study lectures towards a more integrated mix of product/service design. A lecture on ‘Industrial Design’has been added to provide a more integrated view of product/service design (Figure 1), whilst ensuring the primary focus remains on engineering design. There has also been an increasing focus, with the addition of a lecture on inclusive design, on addressing design issues that require the students to have a greater awareness of users and their capabilities. This provides both an opportunity to introduce the principles of inclusion, by describing sources of diversity in the user population, and to present a range of design tools to introduce users, personas and simulators into the design process. This reflects the author’s Table 1.
Course structure for first year product design course.
Pre 2004 Course Structure
Post 2007 Course Structure
Introduction Clarification of the Task Conceptual Design Creativity in Design Case Study – Fire-fighter Systems Case Study – Space Systems Case Study – Electrical Appliances Introduction to Exercise Exercise Feedback
Introduction Clarification of the Task Conceptual Design Embodiment Design Systems Design Industrial Design Inclusive Design Introduction to Exercise Exercise Feedback
Industrial design
Satisfaction Product must be life-enhancing Usability
Technical design
Product/ Service design
Product must be easy to understand and use Functionality Product must work and be safe and economical to use
Figure 1. Teaching integrated product/service design.
Ergonomics for engineers
Figure 2.
487
Concepts for loading two bicycles onto a people carrier.
research interests and has enabled further focus on user interaction and ergonomics, assisted by encouraging the students to reference the ‘Inclusive Design Toolkit’ (Clarkson et al., 2007) online. Following the introduction to the design exercise, students are expected to spend up to 20 hours developing a conceptual design to meet the needs described in the brief. Solutions are marked for their feasibility, practicality and fitness for purpose. Figure 2 shows concepts for a predominantly mechanical design task, typical of many of the earlier projects. Over the years there has been a shift in the type of design problems set to encourage a sharper focus on the details of the concept that influence human interaction.
488
P.J. Clarkson
Figure 3. A concept for a slab lifter.
Figure 4. A control panel for a washing machine. For example, a concept for a ‘slab-lifter’ provides little opportunity for exploring and presenting ergonomic details (Figure 3), while that of a ‘washing device’ does (Figure 4). These changes, implemented over the course of several years, have moved the course from a traditional ‘engineering design’ viewpoint (Pahl et al., 2000) to a product/service design focus with much greater emphasis on users, usability and accessibility (Ulrich and Eppinger, 2007). Whilst it is difficult to assess student response to such changes since no one cohort knows what went before, student survey responses remain favourable when compared to other more technical courses. Further changes are planned to re-brand the inclusive design teaching as usercentred design in an attempt to promote it as just as an element of normal design practice rather than something that is ‘added on’. This mirrors attempts by many educators to promote ‘design for manufacture’ as an integrated part of the normal product development process.
Final year product design course The Design Case-Studies course aims to explore the multi-disciplinary nature of engineering design and demonstrate the importance of considering user needs; illustrating these issues through case studies of form, component and system design. On completion of the course students should appreciate the importance of multidisciplinary systems design and the role of aesthetics and ergonomics in engineering design. The ergonomics portion covers topics such as: definitions, interfaces; physical ergonomics; design strategies; cognitive ergonomics; affordances and constraints; mappings, conventions and metaphors; and inclusive design. The course is not intended as training for future ergonomists, rather it attempts to ensure that engineers appreciate the important perspective they bring to design.
Ergonomics for engineers
489
Figure 5. A further concept for loading a bicycle onto a people carrier. The course comprises two case studies. The first, the design of an inhaler test machine, focuses on function-driven machine design. The second, the design of a cycle lamp, focuses on form-driven design. In the later case, ergonomics and aesthetics form an explicit part of the learning process, accompanied by lectures and student assignments. A studio-type design environment is created where students are encouraged to present and discuss their work in the presence of professional designers are and marks are given for good ergonomic design. The two very different case-studies test the students’design skills in very different ways, but in doing so allow them to focus on different aspects of design. The quality of the designs produced is always very high, often far beyond the students’ own expectations. It is clear that a deep understanding of the challenges of complex systems design and user-centred design is achieved in a relatively short time and this course remains one of the most highly rated by the students in the fourth year.
Discussion Teaching design to a class of over 300 first year engineers remains a significant challenge. For the majority it is their first introduction to the subject and the need to provide a framework for their initial attempts at product design is paramount. The systematic approach described by Pahl and Beitz (Pahl et al., 2006) serves well in this respect, but is rooted in the traditions of engineering design, lacking proper reference to modern thinking in ergonomics and user-centred design. In developing the Product Design teaching at Cambridge there has been a conscious move to expand the first year course, primarily to raise awareness of wider design issues. This has led to the introduction of lectures on industrial design and inclusive design, and an increased focus on user-centred design when defining the product design exercise. Such changes on their own are unlikely to produce better designers. However, the priority here is to enthuse the students about design (Figure 5) and to help them see it as an essential (and natural) partner to the engineering science that pervades the early years of their degree.
490
P.J. Clarkson
More radical changes have also been made to the final year design teaching to expose students to some of the approaches more commonly found on Product (rather than Engineering) Design courses. At the heart of these changes is the desire to promote design as an integrated activity where ergonomics and user-centred design are part of good design practice rather than something that is added only after the engineering design has been completed. This provides a springboard for those who are thinking about product design as a career option and each year we encourage a number of students to apply to join courses, such as the MA course in Industrial Design Engineering at the Royal College of Art. The introduction of elements of ergonomics into the design teaching at Cambridge has been accomplished at the same time as introducing elements of systems thinking. Both processes have, in some respects, proved challenging. Students, particularly those without industry or design experience, can struggle to appreciate the importance of both concepts. Conversely, teaching inclusive design to experienced designers draws naturally on their own experiences of design and the ideas are more readily understood. The use of design examples, particularly if personal to the lecturer, is critical in this respect. Indeed, it is this author’s view that only if key examples are well known to the lecturer, can they be used to inspire the students. In summary, it is vital that engineering courses introduce ergonomics to engineers. However, it is equally important that this is achieved as part of an integrated approach, based on clear principles, grounded in a real understanding of the design need and in the context of examples of effective product design and appropriate student design exercises. Above all else, engineers must be taught to appreciate and respect the professional skills offered by ergonomists and manufacturing engineers in the context of an integrated design activity.
References Clarkson, P.J., Coleman, R., Hosking, I. and Waller, S. 2007, Inclusive Design Toolkit. Engineering Design Centre, (University of Cambridge, Cambridge). Available at: http://www.inclusivedesigntoolkit.com/ Engineering Council 2005, UK Standard for Professional Engineering Competence (UK-SPEC): Chartered Engineer and Incorporated Engineer Standard, Second Edition, (Engineering Council, London). Engineering Council 1997, Standards and Routes to Registration (SARTOR), Third Edition, (Engineering Council, London). Finniston, M. 1980, The Finniston Report. Engineering our Future: Report of the Committee of Inquiry into the Engineering Profession, (HMSO, London). Hamilton, J. 2000, The Engineering Profession, (Engineering Council, London). Pahl, G., Beitz, W., Feldhusen, J. and Grote, K.H. 2006, Engineering Design: A Systematic Approach, Third Edition, Wallace, K.M. (ed.), (Springer-Verlag, London). Ulrich, K. and Eppinger, S. 2007, Product Design and Development, Fourth Edition, (Mcgraw-Hill Higher Education, Columbus, OH). Wallace, K.M. 1993, Teaching engineering design in the new four-year course at Cambridge University, International Journal of Mechanical Engineering Education, 21, 233–246.
THE TRIALS AND TRIBULATIONS OF TRYING TO APPLY ERGONOMICS IN THE REAL WORLD∗ M. Kenwright1 & D. Pennie2 1 2
ALSTOM Transport Greenstreet Berman
Human Factors/Ergonomics (HF/E) can be a difficult concept both to understand and to apply. It often comes across as a woolly or fuzzy topic that seems far removed from the more concrete and technical issues that engineers and business people deal with on a day to day basis. The methods used to explore errors and unsafe behaviours do not always lead directly to effective change and there can be many barriers to the successful application of a HF/E approach. This paper explores the experiences of an individual trying to promote and apply HF/E principles within an organisation over the last 3 years. The paper draws out lessons learnt from this experience and provides practical advice for others trying to do the same. It considers how to capitalise on successes, to create a sustainable focus on using HF/E principles and approaches to drive improvements across the organisation, whilst overcoming the predictable obstacles and barriers to such work.
Introduction To the HF/E specialist, academic or government agency; following good practice guidelines or applying recommendations that concern HF/E appear on the face of it to be straightforward. They often seem like the obvious and common sense thing to do. There are also additional incentives, which should encourage the uptake of HF/E for example: • • • •
Regulatory function (guidance and inspection); Moral obligation (welfare of workers); Reducing the likelihood of costly accidents and incidents; Improving business performance (workers work more effectively and efficiently).
Despite these powerful drivers HF/E does not always appear to be adequately taken into account in the way work is managed. HF/E principles are not always fully embraced by industry and recommendations to reduce error occurrence and enhance performance are not always adopted. ∗
Paper not subject to review
491
492
M. Kenwright & D. Pennie
This view is largely anecdotal but there is also more substantial evidence which lends support to this belief, for example, the re-occurrence of major accidents where a failure to address issues HF/E was a root cause (Baker et al., 2007). In addition, when organisations do bring in outside support, to address human performance issues (error and behaviour), HF/E knowledge and techniques can sometimes be mis-applied, for example: • Final decision on interventions to address HF/E issues, are made by managers based on influences from other functions within the organisation; • The HF/E focus is on changing individual behaviour of workers without reflecting on true root cause such as organisational safety culture and leadership; • It is used to assure or approve work systems rather than inform design. It is not reasonable to argue that this situation exists because industry is simply resistant to attempts to apply HF/E knowledge, principles and approaches. Organisations undoubtedly want to achieve a reduction in accidents; improve business performance and have a happy and productive workforce. A more likely conclusion is that, in reality, the application of HF/E is less straightforward than might at first appear. So why does this apparent dichotomy between HF/E principle and application exist? Why can something that appears, from one perspective, like a straightforward and common sense thing to do on the other hand seem complex and difficult to implement? To get closer to the truth it is important to acknowledge that conceptually HF/E poses some significant issues to non specialist and, some would rightly claim, specialist as well. It often comes across as a woolly or fuzzy topic that seems far removed from the more concrete and technical issues that engineers and business people deal with on a day to day basis. (HSE Research Report 149, 2003). The discipline of HF/E is also very diverse. This can be a weakness because with such a broad spectrum it can be difficult to: define what HF/E is; what it represents and where its boundaries begin and end (Salas, Eduardo 2008). HF/E does not always provide individual discreet solutions and often multiple interventions are required. Some of these interventions may require change outside of the immediate domain of the problem area, with an impact on the organization as a whole. The difficulty of integrating HF/E also means it is too frequently a bolt on addition, rather than being central to business thinking. This can challenge individual managers and departments already stretched to deliver to time and cost. This situation can be exacerbated because HF/E is rarely applied at the right time, at project outset, arriving too late to be effective. This means that HF/E is seen, too often, as a threat to project completion and therefore viewed in a negative rather than positive light (Soekkha, 1997). Integrating HF/E into management systems can also take time and may require a seismic shift in organisational culture and attitudes toward safety; and changing such collective values can be very difficult (Reason and Hobbs, 2003).
The trials and tribulations of trying to apply ergonomics in the real world
493
Finally it can be difficult to link the application of HF/E to demonstrable improvements; making it a challenge to win support from senior management or present the business case for significant change. The objective of this paper is to provide greater clarity to the trials and tribulations of applying HF/E principles in the real world. It does this by exploring the experiences of a non HF/E specialist, in trying to promote and apply HF/E principles within his organisation over the last 3 years. Its aim is to provide greater insight into: • The steps that led to a greater adoption of HF/E; • The successful outcomes; • The lessons learnt – what are the barriers/enablers that may hinder or help the facilitation of an HF/E approach; • How HF/E be sustained. The background to why a more proactive effort to apply a HF/E was adopted is explored under the next heading.
Background The non HF/E specialist, interviewed for this paper, is a senior safety professional working in maintenance sector of the UK overland rail industry. They have since gone on to complete a master’s degree in safety engineering with a final dissertation on: “human error in railway maintenance”. The organization is a provider of new vehicles, maintenance, signalling and parts to the UK rail industry. They designed; built and maintain the Pendolino fleet for Virgin Trains on the West Coast Main Line. The organization, as part of their maintenance function, employs around 2,500 people across the UK In 2005 the interviewee became concerned about repeat incidents and the fact that the recommendations to address them were seemingly ineffective. Following a review of investigation reports, they found that the majority of incidents (over 60%) were caused by human error, with a root cause identified as an HF/E issue. The recommendations arising from the investigations predominately concerned: briefing; re-training or discipline of the maintainers involved. For example: • Incident: (26th May 2004) “incorrectly fitted pantograph head”; • Description of incident: “during train prep driver noticed pan assembly not seated correctly”; • Root cause and comment: “Human factors issue (maintenance function) pantograph head bolts were fitted the wrong way round. Start of shift, a brief issued to raise awareness. No design changes made”. Following repeat incidents they concluded that the recommendations were not effective or adequately addressing the root cause of the incident. This encouraged them to
494
M. Kenwright & D. Pennie
find out more about HF/E and how it might help to tackle issues that were perceived to be impacting on business and safety performance. This experience, and their reflection on it, is detailed in the Findings section.
Findings This section provides the outcomes from a series of interviews with a non HF/E specialist. These interviews explore the experience of applying a HF/E approach and cover the following key topics: • • • •
The steps taken to apply HF/E; Outcomes; Lessons learnt; Sustaining HF/E.
The steps taken to begin to apply a HF/E approach The foundation to improve the application of HF/E approach was a reporting system that: • Captured the occurrence of Hazardous Incidents (HI); • Described the events leading up to these incidents; • Commented on the causes of the incident and made recommendations to address the problem. This reporting system made it possible to identify that over 40% of incidents were a reoccurrence of previous events. It also helped to highlight that recommendations did not tackle the issues that had caused these events. Another important factor was that this apparent failure was identified by an individual unwilling to accept this situation because these repeat incidents posed a risk to the organisation and the wider public. They correctly concluded that the recommendations, and investigation process that generated them, was therefore ineffective; particularly the ones that focused on briefing or re-training the individual directly responsible for the incident. The second key step was that this individual wanted to find out more about human error and improve their understanding of HF/E. This was because it was the classification used to describe the root cause of the majority of the HIs. They began to read more about the subject learning about error occurrence through publications such as: Managing Maintenance error by James Reason and Alan Hobbs. The key message was that accidents and incidents are not random events beyond the control of management. Errors are the consequence of how the work system has been designed and are shaped by: equipment; task design and even the overall culture of the organisation. They concluded that if this was the case it was important to find out what other industries were doing to control and manage error. They found that in aviation and the nuclear sector there was a higher level of importance placed on addressing human error and these industries tended to employ HF/E specialists. The resolved to meet with a number of HF/E consultants to consider what more could be done.
The trials and tribulations of trying to apply ergonomics in the real world
495
The third step was to gain approval from senior management to secure the resources to employ HF/E expertise from outside the organisation. The interviewee stated that because they were part of the senior management team it was relatively easy to convince others within the organization of this requirement. It was also self-evident, that the repeat incidents were costing the business money and were a significant risk to the organisation. Current interventions were not working and it was, therefore, necessary to take a fresh approach. The fourth step was to select a suitable candidate to provide this support. The HR director personally knew an HF/E specialist, working within the nuclear industry, who had been successful at helping other organisations. The specialist met with the senior management who were won over by his approach, and the training methods he proposed. The training was very practical and engaging and provided a very powerful and clear message into how organizational factors and behaviours can lead to errors. The fifth step was how to launch the initiative. A team comprising: the safety professional; Traincare Centre Managers and HR director was established. This team then attended a two-day HF/E training course. Following the training a meeting was the held to: • Develop a brand for the new initiative (Human Error Prevention); • Plan the approach and identify ways to enable and embed the initiative; • Decide on the type of training to complement the branded initiative and how this would be rolled out across the organization. The sixth step was to pilot the scheme at the Glasgow maintenance depot and then begin the programme of work to integrate HF/E principles into safety management. This involved: • Selecting an HF/E champion who was enthusiastic about supporting the initiative and who were respected by their peers within the organization; • Improve knowledge about human performance and why errors and violations occur across all levels of the organisation (1 day course for maintainers 2 day course for managers and HF/E champions); • Create a more just reporting culture and change the incident investigation process – develop the “Balance of Error Tool”: • Provide all employees with a handbook about human error “something for the top pocket” – endorsed by the Customer Director; • Introduce a number of tools and techniques to help maintainers think about their behaviour and prevent error from occurring (Self Check, Peer Check Pre Job Briefs). In conjunction with the integration of HF/E into safety management, a number of other systems and processes were applied to improve the efficiency of how work was carried out. For example: using techniques like: LEAN manufacturing; 6-Sigma and “Pit stop maintenance”. These techniques seek to improve the design Systematic approaches used to help improve the way tasks are designed
496
M. Kenwright & D. Pennie
of tasks and eliminate wasteful practices, such as walking long distances to collect tools and equipment. This led to a number of interventions, for example: • Introducing line side tools and vending machines that contain regularly used components and parts; • Ensuring personnel, components and equipment are in place so work can begin as soon as the rail vehicle arrives in the pit lane; • Reviewing how long a particular maintenance task takes to complete, ensuring that maintainers have sufficient time to finish the required work before the rail vehicle needs to return to service; • Standardising procedures across different depots. These interventions have helped to increase the time maintainers have to carry out inspection and repair work; which has helped to make the maintenance work more reliable and robust.
Outcomes One of the difficulties of applying an HF/E approach is demonstrating the measureable benefits that it can bring and justifying the cost of intervention. The cost of: consultancy; training and developing materials, excluding the time of employees away from work, was estimated to be between £40-60,000. The occurrence of an incident is measureable but there is no easy way to show that a potential incident has been prevented. It is also difficult to provide clear evidence that people’s outlooks and behaviour have changed. The organization has shown, however, improvements in terms of a reduction in injury rates and also train reliability. It is difficult though to show a clear casual link between these improvements and the integration of HF/E. This is because these improvements are also likely to be a product of other business interventions, particularly those to enhance the efficiency of how work is carried out. Anecdotally the key reported outcome, where it easier to establish a link, is the increase in incident reporting. This is because management commitment to improving the reporting culture has meant that maintainers have shown a much greater willingness to highlight areas of concern. This has enabled the organization to identify weaknesses in defenses that might allow errors to propagate as events or Hazardous Incidents. For example, reporting has led to a number of recent changes to the way certain maintenance tasks are carried out. Possibly the strongest confirmation of the benefits for the initiative is that the senior management team have not asked for evidence or questioned the worth of the cost of the HF/E training or consultation. In hindsight the interviewee stated that one way they could have measured the success of the initiative was by benchmarking and using different indicators to measure the before and after effects of the intervention. For example: conducting a safety culture survey to assess worker attitudes towards safety.
The trials and tribulations of trying to apply ergonomics in the real world
497
Lessons learnt Several key lessons were identified, from the interview, and the following insights might help provide guidance to others looking to apply a HF/E approach and/or integrate HF/E principles into safety management. According to the interviewee the most important enabler is the backing of senior management and their willingness to listen, and be convinced, that HF/E will be good for the business. Without this it is very unlikely that initiatives to integrate HF/E principles will be successful. The interviewee thought, that for his organization, the support was more forthcoming because he was part of the senior management team. This meant the other senior managers, who agree and authorize the required resources to implement the initiative, were prepared to be guided by his judgment. The HR Director was also had a good ally who helped to support the initiative. The decision to integrate HF/E in safety management was also facilitated by the arrival of new management from the manufacturing division of the organisation in 2005. They brought with them different ideas, initiatives and processes with an origin in manufacturing. These were equally applicable to enhancing maintenance performance. The existing management had always been aware of the issues impacting on the business but had suffered from inertia and an inability to change their management style. It is also important to provide managers (senior and local) with training before the workforce (maintainers). Without developing the knowledge of managers, their attitude towards HF/E principles might be less positive and this can undermine the training delivered to other groups. It was particularly important to achieve a fundamental change in the understanding of the Train Care (Depot) managers towards HF/E. This is because they make the decisions about whether to undertake disciplinary procedures. Exposure to HF/E training and the need to apply the Balance of Error Tool (BET) created a more ‘just’ culture. This changes the focus of the event so that the investigation is no longer centered solely on the role of the individual. Instead it seeks to explore whether the error was induced by the organization. Developing the understanding of the senior and local management also helped to overcome a key barrier to improved HF/E integration, the suspicion of workers and union representatives. Workers can be understandably anxious about change and how it may affect the conditions of their future employment. Concerns may also have their origin in prior experience. Often workers have seen new initiatives come and go without tangible benefit to them directly. Shortly after the “Balance of Error Tool” (BET) had been introduced an incident occurred that required an investigation. To re-assure workers that there was genuine desire for change; union representatives were invited to attend and observe how the new approach worked in practice. The maintainer, involved in the incident, was re-assured that there was no assumption that they were to blame and there were other factors that needed to be considered. The new process asks the question: “Would (or has) a similar person done the same in the same situation and was the error that led to the incident
498
M. Kenwright & D. Pennie
induced by the organisation”? The investigation revealed that there were lots of different influences that affected the judgment of the maintainer. These originated from both the organisation and outside influences in the maintainer’s personal life. Rather than seek to blame the maintainer, the organisation provided him with more support. They also identified how they could reduce time pressures on the maintainer. Maintainers were won over by the no blame approach. The application of the BET provided evidence that management had introduced new techniques to create a more open and ‘just’ reporting culture. Management were: “walking the talk”.
Future plans to sustain HF/E A key difficulty is sustaining improvements and keeping HF/E at the forefront of people’s minds. This is especially hard when encouraging people to keep reflecting on their behaviour. Outside agents like HF/E specialist can “ignite the flame of interest” but they move on and then the challenge is for the organisation to maintain impetus. To help ensure momentum is sustained, the organisation plans to introduce a number of measures, for example: • Continue to recruit human error prevention champions; • Create a steering group to ensure a consistent approach is being applied across all the maintenance depots; • Conduct organization wide trending and error analysis; • Carry out management coaching – ensure maintainers are using the error prevention tools and techniques and if not explore the reasons why; • Provide feedback so maintainers can see more tangible evidence of how their own performance can improve business performance.
Conclusion The interview has helped to provide more understanding of the required stages or steps to achieve a greater integration of HF/E in safety management. The main factors that identified from the interview, that were considered to be critical to success, were: • The willingness of an individual to champion the HF/E approach; • Total buy-in from senior management and support from local managers; • The availability of resource to bring in external and trusted expertise. Without all these factors in place, it is likely to be difficult to successfully integrate HF/E into safety management or improve the safety and effectiveness of work. There are also additional barriers to the uptake of HF/E but evidence, from the interview, suggests that these are not insurmountable and the best way to achieve ‘buy-in’ is to: • Enhance knowledge across the organisation;
The trials and tribulations of trying to apply ergonomics in the real world
499
• Demonstrate HF/E principles in action, for example, radically changing the approach to incident investigation; • Secure quick wins that solve safety and operational issues and show the benefit of HF/E to management, workers and the organisation as a whole. In addition because HF/E can come across as a nebulous and fuzzy topic, it is important to brand it, with a meaningful ‘strap-line’, so it is: bounded; relevant and more easily understandable to the organization as a whole. Sustaining HF/E and challenging behaviour is a “constant battle”, particularly when a programme of training has come to an end. It is, therefore, important to seek new ways to keep initiatives alive. It is also important to embed HF/E principles and knowledge in a number of HF/E champions and in key positions, such as, site/depot managers. If one person leaves then momentum is retained by others across the organisation.
References Baker, J et al., 2007 Independent Safety Review Panel, the report of the BP U.S. refineries. HSE Research Report 149 2003 – The promotion of human factors in the onshore and offshore hazardous industries (HSE Books). Salas, Eduardo. 2008, At the Turn of the 21st Century: Reflections on Our Science, Human Factors 50 351–353. Hans Soekkha. 1997, Aviation Safety Human Factors, System Engineering, Flight Operations, Economics, Strategies, Management (Published by VSP). James Reason and Alan Hobbs. 2003, Managing maintenance error – a practical guide, (Ashgate Publishing). William B et al., 1997, The Case for Human factors industry and government: Report of a Workshop (Published by National Academies Press). Moray, Neville. 2008, The Good, the Bad, and the Future: On the Archaeology of Ergonomics, Human Factors 50 411–417.
TRANSPORT
A PRELIMINARY INVESTIGATION OF A LOW-COST METHOD FOR PREDICITING THE DISRUPTION OF GLANCES TOWARDS IN-VEHICLE INFORMATION SYSTEMS A. Irune & G.E. Burnett School of Computer Science and IT, University of Nottingham, Nottingham, NG8 1BB, UK For in-vehicle information systems (IVIS) such as navigation and communication, designers must aim to understand user requirements early and evaluate these systems in an iterative fashion. Given that glance duration appears to be the most promising ocular-based indicator of attention or exposure to risk, a study was conceived to develop a preliminary approach to predicting secondary task demands, based on primary task performance, with specific focus on glance duration. Ten participants performed a range of secondary tasks (ST) while attending to a primary loading task (PLT) and a positive relationship was found between the number of PLTs missed in a sequence and the mean ST glance duration (p = 0.01).
Introduction Technological advances in computing and communication technology over the years have enabled the integration of increasingly sophisticated devices in cars, for both driver support (e.g. navigation aids) and infotainment (e.g., news and email) (Irune and Burnett, 2007). As established by past researchers (Wierwille 1987; Rockwell, 1988; Lansdown, 1997), the design of these in-vehicle information systems (IVIS) interfaces vary significantly and user responses can be affected by their design. IVIS tasks are essentially secondary to the primary driving tasks associated with safe control of the vehicle within the surrounding traffic environment. Consequently, given the accessibility of IVIS and the likely frequency with which drivers will make use of the technology, designers must aim to understand user requirements early and thoroughly evaluate IVIS interfaces in an iterative fashion. IVIS typically use complex visual displays -it is therefore critical for designers to understand the visual demand of their interfaces and the subsequent effect on driver performance. In recent years, there has been a revolution in the way products are developed. It is no longer accepted practice to wait until the end of development to evaluate a product (Jacko et al., 2003). Moreover, the automotive industry is constantly seeking cheap and cost-effective ways of evaluating and improving new and existing designs. In this respect, formalised methods, such as the occlusion technique (ISO, 2006) and the lane change test (ISO/WD, 2005) enable surrogate measures of visual 503
504
A. Irune & G.E. Burnett
demand to be captured. However, practical limitations with these methods present real drawbacks to the automotive industry (e.g. need for proprietary software and hardware). In addition, none of the methods currently available provide a surrogate measure of glance duration, that is, the mean length of glances made away from the forward road scene. A method that predicts glance duration is vital for identifying interfaces that encourage prolonged exposure to risk (i.e. long glances away from the road scene). Glance duration has been noted by many authors to be an important ocularbased indicator of attention. Rockwell (1988) found there was a correlation between exposure to risk and average glance duration. Wierwille (1995), Green (1999), Tijerina (2000), Farber (2000), Greenberg (2000) have also suggested that, in relation to in-vehicle tasks, the total number of glances or chunks needed to complete the secondary task is not critical. Of greater importance is whether the task can be accomplished in separate incremental steps with glance durations typically less than two seconds. This paper describes a preliminary approach to predicting secondary task demands based on primary task performance, with a specific focus on glance duration. An empirical study is presented which provides an initial indication of the potential for the method.
Novel method logic This approach draws on the paced, visual alternating aspect of the occlusion method (i.e. switching from a fixed period of secondary task engagement to a period of nonsecondary task involvement). It differs significantly from occlusion though, in that a primary loading task (PLT) must be timeshared with secondary task engagement. By including two competing tasks, the PLT approach aims to prevent people rehearsing their suspended goals, therefore ensuring the disruptive effect of the PLT/secondary task cycle. As shown in fig 1. Subjects will begin the PLT at the start of the trial, after a short period they engage with the secondary task. The PLT continues to run simultaneously while the subject attempts to perform the secondary task. Shortly after the secondary task is completed, the PLT is terminated. The PLT was designed to include key components of real driving (visual-spatial search and working memory tasks), as it has been recommended that the nature of
Figure 1.
Method design.
A low-cost method for predicting the disruption of glances towards IVIS
505
any task to replace the occluded period should closely resemble the driving task (Trafton et al., 2003). This study ensured a systematic manipulation of the visual demand of the secondary task whilst assessing the consequential effects on the performance of the PLT. Of primary interest as a surrogate measure of glance duration was the number of PLT screens missed. The duration of glances made towards the secondary task display was also measured in order to make a preliminary assessment regarding the validity of the approach. Ultimately, it was hoped that the noted effects on PLT performance could be used as a measure for the secondary task demand.
Method Ten participants were recruited for this study. Participant characteristics were as follows: • • • •
Mean age 31.7 (range 22–40) All held full driving licences for at least 2 years Mean age of holding driving license = 11.7 years (range 3–28) Mean number of miles driven in last year = 8,600 (range 3,000–20,000)
The study lasted approximately half an hour and required participants to undertake five secondary tasks (see below). In a repeated measures design, each participant undertook five variants of each secondary task making a total of 25 tasks for each trial.
Procedure Participants were requested to complete a consent form and pre-study questionnaire. They were then introduced to the equipment and the two tasks they would be performing during the trial – the PLT and the secondary task (ST). Participants were fully trained in both tasks. Participants were informed on the prioritisation of the primary and secondary task. The primary task was to be given the most priority for the duration of the trial with the aim of maintaining error free performance through the duration of the trial. However, the secondary task was to be completed as quickly as possible.
Primary Loading Task (PLT) Participants were requested to alternate their vision between the PLT and the ST. The PLT occurred as part of a PowerPoint slide show that appeared on a 17 monitor directly in front of the driver (see Figure 2). An auditory beep occurred each time a new PLT screen appeared (once per second). For the PLT, participants were asked to search and compare two blocks containing 2 × 2 squares with either a cross or noughts (‘X’ and ‘O’) in each square, Using a pair of push buttons located on either side of the steering wheel, participants were asked to indicate, which block contained more crosses; left box (left push button) or right box (right push button)
506
A. Irune & G.E. Burnett
Primary Loading Task
Left LED
Right LED Left button
Secondary Task
Right button
Figure 2.
Showing task set-up and components.
Verbal Response “CLOSED”
Figure 3.
Secondary task (example of a 3 second task).
(see fig. 2). In piloting the PLT, it was established that this simple visual-spatial task was consistently achievable in the one second period available.
Secondary Task (ST) The secondary task was designed to mimic real world secondary in-car tasks (visual and cognitive interaction requiring some form of input and output). The secondary task utilised a display placed in the centre console of the dashboard (see fig 2). Importantly it was designed to promote glances of varied lengths through a series of squares presented to the driver. These lengths were 1, 1.5, 2, 2.5 and 3 seconds respectively. Each square represented a time slot of 0.5 seconds therefore the minimum number of squares was two while the maximum was six. Each square contained a piece of information relating to an arbitrary junction. A collection of these squares formed a series which ended with an ‘X’ CLOSED or an ‘O’ OPEN. Participants were instructed to indicate if the named junction highlighted by the series of squares was open or closed by saying, “open” if open and “closed” if the junction was closed (see fig. 3). Each task would show a series of squares presented one after the other
A low-cost method for predicting the disruption of glances towards IVIS
Figure 4.
507
Scatter plot of all measures against tasks.
every half a second until the last square which contained this piece of information was displayed. The order of the timing for the secondary task was randomized to ensure that participants could not predict the length of each task. In this respect, an attempt was made to manipulate drivers’ visual attention in a controlled way across these varied times (i.e. 1, 1.5, 2, 2.5 and 3 seconds). Referring to figure 3, from top to bottom, each rectangular box to the left of the table represents what would be seen on the ST screen at 0.5 second intervals across a 3 second ST task. At the start of each trial, participants were required to perform the PLT for approximately five seconds. After which participants were signalled to begin the secondary task. Participants were instructed to aim for completing the ST as quickly as they could whilst maintaining error free performance with the PLT. The ST was repeated two times, which ensured participants had a total of three attempts to perform the task. Each trial was video recorded and a frame by frame video analysis was performed. Measures for glance frequency, glance duration, number of PLT screens missed, number of PLT screens missed sequentially and Number of non-responses to PLT screens were obtained. Mean values were also calculated for the above measures.
Results The results of the study show a clear relationship between the length of the secondary task and measures for glance frequency, glance duration, PLT non responses and the number PLT screens missed in a back-to-back sequence (see fig 4). The x-axis presents the results for all ten participants across all five task conditions. Each
508
A. Irune & G.E. Burnett
Figure 5.
Secondary task and mean glance and PLT measures.
Table 1. Expected glance duration versus mean total glance duration with standard deviation in brackets. Expected glance duration (seconds)
Mean total glance duration (seconds) (Stdev)
1 1.5 2 2.5 3
1.01 (0.34) 1.23 (0.27) 1.39 (0.25) 1.58 (0.24) 1.93 (0.36)
condition is represented as follows: Condition 1 (1 second task) is represented by values 1–10 on the x-axis, Condition 2 (1.5 second task) is represented by values 11–20 on the x-axis, Condition 3 (2 second task) is represented by values 21–30 on the x-axis, Condition 4 (2.5 second task) is represented by values 31–40 on the x-axis, Condition 5 (3 second task) is represented by values 40–10 on the x-axis. Figure 5, further highlights a positive trend in the relationship between the mean values for all 10 participants for the number of PLT screens missed in sequence, the non-responses to PLT screens and the mean glance duration in relation to the length of the secondary task. To provide an indication of the relationship between mean glance duration and expected glance duration (i.e. secondary task allocated time 1–3 sec), a correlation analysis was performed on the data. Standard deviations are also shown in brackets under ‘mean total glance duration’ in table 1).
A low-cost method for predicting the disruption of glances towards IVIS
509
PLT missed in sequence vs mean glance duration 3
PLT missed in sequence
Miss_Seq Linear (Miss_Seq) 2.5
2
1.5
1
0.5
0 0
Figure 6.
0.5
1
1.5 2 Mean Glance Duration
2.5
3
Mean glance durations against PLT missed in sequence.
Further correlation analysis for each glance behaviour measure showed that participants mean glance duration possessed the strongest correlation with expected glance durations (R = 0.737) across the different tasks type (1, 1.5, 2, 2.5 and 3 seconds), with correlation significant at the 0.01 level (2-tailed test). A statistically significant correlation was also found between the mean values for glance duration and the number of PLTs missed in a back-to-back sequence (R = 0.563), with correlation significant at the 0.01 level (2-tailed test). An analysis of the relationship between the number of PLTs missed in sequence across all tasks and participants’ mean glance duration is presented in figure 6 above. Mean glance durations of 1.5 and 2 seconds have been highlighted on the x-axis. When these are further extended to the y-axis, it indicates the number of PLTs missed in a back-to-back sequence, equivalent to mean glance durations of the aforementioned times. The values of 1.5 and 2 seconds have been highlighted as critical values due to the fact that, past research have found that glances lasting longer than about 2.0 seconds made to IVIS are not acceptable while driving.
Discussion The results of this study highlight a strong correlation (R = 0.99) between the desired glance duration i.e. 1, 1.5, 2, 2.5, 3 seconds and average glance duration. Indicating that the desired glance behaviour was achieved, that is, drivers glanced for progressively longer periods for task types designed to promote progressively
510
A. Irune & G.E. Burnett
longer glances. Although both sets of times (desired glance and actual glance duration) were not exactly identical, they provided a variation of times across a range that could be used to interpret and understand the effect of a variation of glance times on the primary task. Further analysis of the data showed a significant correlation between the PLT performance in terms of PLT screens missed in a back-to-back sequence, and the mean secondary task glance duration (R = 0.563). This significant correlation indicates that the duration of glances made to the secondary display have a statistically significant relationship with the performance of the primary task, in terms of how many PLT screens were missed in a back-to back sequence. The analysis also highlighted the degradation in primary task performance across this specific measure, as a result of sustained visual attention needed to achieve a secondary task. Gelau and Krems (2004) highlighted certain constraints identified in experimental research which secondary tasks must meet in order to be acceptable while driving. One example is the duration of single glances away from the road. Bhise, Forbes and Farber (TRB, 1986) state that based on speed and travel distances of a moving vehicle, a single display glance greater than 2.5 seconds is inherently dangerous. The British Standards Institution Guidelines published in 1996 recommend that a single glance should be not more than 2 seconds so that it does not affect driving (Green, 1999). The Battelle Guidelines suggest that the average glance time not exceed 1.6 seconds for a task to be considered safe for on-road use (Green, 1999). It is generally agreed that glances lasting longer than about 2.0 s. are not acceptable while driving (Gelau and Krems, 2004). This approach could potentially provide upper limits for glance duration based on PLT performance, thereby highlighting interactions which promote glance behaviours that are unacceptable while driving. Variability in the results obtained from the study was highlighted by the scatter graph (see fig 4) and by the standard deviation values of actual glance duration (Table 1). This variability can be attributed to the different strategies employed by participants. It is important to note that participants had three attempts to perform the secondary task whilst also engaging with the PLT. It was observed that in some cases participants placed more emphasis on the secondary task compared to the primary task, focusing on accomplishing it as soon as it begun. Others tried to pre-empt the sequence of the PLT, so as to return to it in time to achieve it. In some cases, participants neglected the secondary task completely with the view that they could achieve it on the second or third attempt. It was also observed that some participants tried to pre-empt the length of the secondary task by intentionally missing the first sequence of squares but gaining an impression of the length of the sequence via peripheral vision. This increased their chances of returning to the repeated sequence at the appropriate time the second time round. Although the results obtained were desirable, minimising the strategies adopted by participants would have produced less variability. This could have been achieved through a more stringent protocol procedure (e.g. highlighting the importance of achieving secondary task on very first attempt, or by simply reducing the number of attempts at achieving the secondary task to one). These promising results provide a starting point for the development of a predictive tool that can potentially provide a surrogate measure for the secondary
A low-cost method for predicting the disruption of glances towards IVIS
511
task demand metric of glance duration without the extensive and laborious processes involved in obtaining glance duration measures from other methods (e.g. simulator studies, road trials). Nevertheless, the current approach will benefit greatly from further refinement. For instance, the method needs to be used with a range of real world in-vehicle secondary tasks to evaluate its usefulness and sensitivity. Tests must also be done to compare results across a range of other human factors methods to establish further the methods validity. Finally, it may be beneficial to consider the method with a range of IVIS that require other modes for interaction, for example, touch and speech. These varied interactions may highlight further issues that may not have been considered by the present approach.
Conclusion Current and future in-vehicle information systems may present drivers with various safety and non-safety related information, requiring drivers to interact with these interfaces for a varied length of time. It is important for designers to evaluate these systems through their development and ensure interactions that require prolonged glance behaviour are either chunked (split into shorter time allocations) or performed while the vehicle is stationary. Results from the preliminary study described in this paper, should facilitate the further development of a novel method that can provide a surrogate measure for glance duration. These initial steps also bring to light an approach to outlining possible lower and upper limits for PLT performance which will equate to specific glance behaviour characteristics. Consequently, the PLT approach has the potential to provide a simple indicator of what is considered acceptable and unacceptable glance behaviour, for interaction with a given IVIS in a driving context. It is strongly recommended that future work addresses the issue of sampling, i.e. a larger, more representative sample must be tested using this method across a range of real world in-vehicle tasks (e.g. satellite navigation task: destination entry, POI entry etc.). Moreover, a road or simulator study is necessary utilising the same exact tasks and same subjects to facilitate an assessment of the validity of the method.
References Burnett, G.E., 2008, Designing and evaluating in-car user-interfaces. In J. Lumsden (Ed.) Handbook of Research on User-Interface Design and Evaluation for Mobile Technology, Idea Group Inc. Farber, E., Blanco, M., Foley, J., Curry, R., Greenberg, J., Serafin, C., 2000, Surrogate measures of visual demand while driving. Proceedings of the IEA 2000/HFES 2000 Congress, San Diego. Gelau, C. and Krems, J. F., 2004. The occlusion technique: a procedure to assess the HMI of in-vehicle information and communication systems. Applied Ergonomics Volume 35, Issue 3, May 2004, Pages 185–187.
512
A. Irune & G.E. Burnett
Green, P., 1999, Visual and Task Demands of Driver Information Systems (UMTRI Technical Report 98-16), Ann Arbor, Michigan. Greenberg, J., 2000, Visual demand while driving, selected results from a Ford Motor Co. driving simulator study. Presented to the SAE ITS Safety and Human Factors Committee. Irune, A. and Burnett, G.E., 2007, Locating in-car controls: Predicting the effects of varying design layout, In Proceedings of Road Safety and Simulation conference (RSS2007), Held in Rome, November 2007. ISO. Road vehicles – Ergonomic aspects of transport information and control systems – Occlusion method to assess visual distraction due to the use of invehicle information and communication systems. Draft International Standard ISO/DIS 16673.2 ISO/TC22/SC 12, 2006. ISO/WD, 2005. Road vehicles – Ergonomic aspects of transport information and control systems – Simulated lane change test to assess driver distraction, Technical Committee ISO/TC 22, Road vehicles, Subcommittee SC 13, Ergonomics. Jacko, J. A. and Sears, A., 2003, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications. Lawrence Erlbaum Associates. Lansdown, T. C. and Burns, P. C. 1997, Route guidance information: visual, verbal or both? International Ergonomics Association 13th Triennial Congress, Tampere, Finland. Rockwell, T. H., 1988, Spare Visual Capacity in Driving – Revisited: New Empirical Results for an Old Idea. In A. G. Gale, et al. (Eds), Vision in Vehicles II, North Holland Press: Amsterdam, 317–324. Tijerina, L., Johnston, S., Parmer, E., Winterbottom, M. D., and Goodman, M. 2000, Driver distraction with wireless telecommunications and route guidance systems. DOT HS809-069. Trafton, J. G., Altmann, E. M., Brock, D. P., and Mintz, F., 2003, Preparing to resume an interrupted task: Effects of prospective goal encoding and retrospective rehearsal. International Journal of Human-Computer Studies, 58, 583–603. Van Winsum, W., Martens, M. and Herland, L. (1999). The effects of speech versus tactile driver support messages on workload, driver behaviour and user acceptance. TNO-report, TM-00-C003. Soesterberg, The Netherlands. Wierwille, W.W., Austin, J. F., Dingus, T. A. and Hulse, M. C. 1987, “Visual attention demand of an in-car navigation display system”. In Proceedings of Vision in Vehicles Conference, Nottingham, UK. Wierwille, W.W., 1995, Development of an Initial Model Relating Driver In-Vehicle Visual Demands to Accident Rate, Proceedings of the 3rd Annual Mid-Atlantic Human Factors Conference, Blacksburg, VA: Virginia Polytechnic Institute and State University, 1–7.
THE EFFECTIVE MANAGEMENT OF USER REQUIREMENTS AND HUMAN FACTORS ISSUES C. Munro & S.R. Layton Human Engineering Limited, Bristol, BS9 3AA, UK This paper provides a comparison of a number of case studies regarding human factors (HF) input during the detailed design stage (RIBA E / GRIP 5) of a number of railway station upgrade projects. The projects involved HF input to the design of new station control rooms, ticket offices and enhancement of other operational areas. The specific aims of this paper are; to describe the process whereby the user requirements and the HF issues were collected, stored and managed; to demonstrate how gaps are filled if HF input was omitted from previous design stages; and to discuss the lessons learnt from each project. The paper describes the differences between user requirements and HF issues and how, on many projects, there can be confusion between the two, and how they are sometimes wrongly amalgamated into one data set. The paper continues to describe the methods used for collecting, storing and managing user requirements and HF issues, and the hurdles that have to be overcome when working as part of a large project team on a real world project. Furthermore, the paper demonstrates the benefits of effective user requirement capture and HF issue management, and describes a number of approaches that can be utilised depending upon the complexity of the project.
Introduction London Underground has a mandatory Human Factors Integration (HFI) Standard (Reference 1) which sets out the requirements for HFI in London Underground projects. This standard outlines the process by which HF should be managed throughout the project and mandates the production of a Human Factors Integration Plan (HFIP) which should demonstrate how the standard will be met during the project. There are a number of mandated elements which the generation of user requirements and the management of a Human Factors Issues Log (HFIL) support. The greatest of these is the inclusion of end user representation in the design specification and validation process. The generation of a user requirements list and an issues list facilitates the process by which it can be ensured that the user requirements are successfully integrated into the design. Network Rail also has a mandatory Ergonomics Integration Standard (Reference 2) which describes the development of an Ergonomics Integration Plan 513
514
C. Munro & S.R. Layton Task Analysis Operational Concept Review
Baseline Studies
Future Task Analysis developed
User Requirements Generation Standards review in relation to tasks
Derive requirements from TA
Produce Draft User Requirements List
Verify Requirements with Users (Workshops)
Requirements Comparison with Design
Requirements Feed into Design
Issues Raised in Comparison Feed into Design
Design which complies with User Requirements and considers all User Issues Raised
Figure 1.
Issues and requirements capture and integration process.
(EIP) but explicitly advocates the documentation of user requirements as part of the project process.
User requirements and issues generation In user-centred design it is vital to have an understanding of user tasks and to involve users in the iterative process of design. This model demonstrates the use of the future operational concept as the basis for the HF process to understand user requirements, generate user requirements, verify those requirements and capture user issues throughout. The requirements set should then be fed into the design process. Compliance against the user requirements set can then be assessed at varying design stages. The earlier the user requirements set are fed into the design the more easily they can be integrated. The authors have found that the generation of a robust set of user requirements and effective issue capture is supported by the key stages outlined in Figure 1. Each of the key stages outlined in Figure 1 are discussed further below.
Task analysis The basis of user requirements capture is the operational concept. The Operational Concept describes normal and abnormal operating states, defines user roles,
The effective management of user requirements & human factors issues
515
minimum staffing numbers, emergency plans, and equipment required for effective station running etc that feed directly into the capture of user requirements. For example, the operational concept may specify that station CCTV should be viewed in the control facility so the requirement would reflect that. The Operational Concept can be used by all members of the design team to gain an understanding of the operations to be performed in the system. The operations to be performed should form the basis of the design rationale: any system should support the operations to be performed otherwise the system cannot be considered ‘fit for purpose’. Baseline studies are conducted to enhance understanding of the tasks conducted currently at the station. Site visits are often necessary to support the baseline studies, and as these are conducted in live operational environments, distractions are not uncommon. Therefore it is beneficial to have semi-structured interviews prepared for site visits in the event of a distraction inhibiting data collection. Any issues noted at site will be later collated so that any negative issues are entered into the HFIL and positive issues entered into the requirements log. The station operational concept and baseline studies would provide the reader with a high level understanding of job roles within the facility which the ergonomist can then build on with the user group and design team to produce a future task analysis for the new system.
User requirements generation A standards review should be conducted with an understanding of the tasks to be performed in the areas under investigation. The review will therefore be more focussed and illustrate where requirements and best practice advice exists to guide the design of that area. Requirements can often be derived from the task analysis by focussing on the equipment, procedures, and environmental considerations which support the task in question. The authors have discovered that in larger system design projects, using a database to hold both task analysis and derived requirement data is useful in tracing the requirement back to the individual task from which it was derived. It is especially useful when task information is altered later in the project cycle as the derived requirement associated with that task can be revisited at the same time that the task is amended. The requirements generated from standards and derived from the task analysis are then merged to form a draft user requirements list. The involvement of users is critical in the development of a system which is fit for purpose. User workshops are held to verify the user requirements set. There is often the feeling from the ‘engineering side’ of the project that users should not be asked what is required as they will propose a ‘gold-plated’ set of requirements. In the experience of the author this is not the case and that most users self-moderate their input. However end-user input can stray when they cannot envisage the end system or when they are not aware of strategic planning by management teams.
516
C. Munro & S.R. Layton
The UAM/ EIM therefore play an important role in mediating user-provided input to the project team and providing strategic-planning information. The requirements set can then be incorporated into the design if the level of requirements is appropriate for the design stage; i.e. high level requirements for room size and adjacency at concept design stage or more detailed requirements at the detailed design stage. However, if the design is complete before the requirements have been generated there can often be conflict during the comparison stage. At this stage, differences between the design and the requirements set are often due to the following: • • • •
The operational concept does not match the proposed design; Review of design against standards highlights a non-compliance; Baseline studies indicate that a current design is not effective; User workshops highlight problems with the design that should be mitigated.
Where these conflicts are noted they are entered into the issues log and recommendations for their amelioration are provided. A user requirement can therefore often be generated as the mitigation measure to resolve an issue. Negotiation with the project team is often required at this stage to identify constraints and to come to a mutually acceptable solution. During the detailed design phase, the use of 3D software modelling to demonstrate a physical manifestation of the user requirements to users has been found by the author to be a useful technique in bridging the gap between architectural drawings and a physical mock-up. Users have noted that the 3D model provides an illustration of the system which requires less interpretation than architectural plans and elevations. It is therefore an effective way of validating user requirements.
Requirements and issues capture tools Requirements and issues capture tools are relatively simplistic and are often a word table or excel spreadsheet. The content of the issues and requirements logs can be different depending on the purpose of the project. As there usually are project-wide requirements and issues logs, the HF produced logs and requirements databases are often aligned with existing project models to allow ease of transfer. A typical issues log structure is illustrated in Figure 2. A typical user requirements log structure is often less complex and comprises a list of requirements with the following column headings: • Requirement Number • Requirement Description • Requirement Source Throughout the life of the project, the user requirements log can be used to check design and build compliance and so may have an additional compliance column. The issues and requirements logs are live documents and therefore version control is an important internal and external management issue. An Excel spreadsheet is
The effective management of user requirements & human factors issues Heading
Definition
Administration Details Issue Number Raised by Source Issue Details Issue Title Issue Description Issue Status Classification Consequence
517
ID number to be entered ad hoc, using 00#, 0#0, #00 format. Person initials. Document/ meeting whereby issue was raised. A simple issue description in title format. Brief outline of issue. OPEN /RAISED / IN PROGRESS / CLOSED. OPERATIONAL / SAFETY / PROJECT. Provides a textual overview of consequence of issue should it occur.
Closure Details Closure Statement
Details on how the issue has been resolved; for example the mitigation measure incorporated in design.
Recommendation Details Rec. No ID number to be entered ad hoc, using 00#, 0#0, #00 format. A brief outline of the recommendation. Rec. Priority ESSENTIAL / HIGHLY DESIRABLE / DESIRABLE / N/A. Responsible Party Implementer of recommendation.
Figure 2. Typical issues log structure.
especially useful in multi-faceted projects, where the HF team are involved in many aspects of the design and other bodies are focussed on smaller areas.
Communication – The art of tenacity! In a large project team it should be recognised by the HF team that they are the voice of the user and that communication of user needs is the biggest contribution the HF team bring to an interdisciplinary project. The communication of issues to the design team can be managed via publication of the issues log, an agenda item for discussion at design progress meetings or an update to a progress report depending on the project management style. The communication is two-way as the design team will then respond to the issue and determine whether it is feasible to close it out in design. The reciprocal communication can therefore be an updated design or a minuted discussion which determines that the issue can be closed out. The HF team will then reflect the close out details in the issues log in order that there is full visibility of decision making and rationale. Requirements communication is most effective in the early project phases as design change to incorporate requirements is minimal and can be subsumed in design development activity. The requirements set can then be used throughout the project life-cycle as a benchmark for the design.
518
C. Munro & S.R. Layton
It is often the case with designers as well as users, that a design concept or potential design change suggestion is more easily conveyed in illustration rather than text. The incorporation of a textual requirement is often difficult for the designer to interpret if they do not have knowledge of the system or the user tasks to be performed by the system. Increasingly the authors find that 3D modelling, as well as design team engagement with HF activity and end-user consultation provides results more quickly and cost effectively that multiple design iterations produced in response to requirements capture. It is therefore important to demonstrate as part of the requirements and issues management audit trail that the issue has been closed out in the final integrated design. Of course not all layouts are technically feasible and therefore there may be some further communication to close out any remaining issues to the satisfaction of all parties.
Benefits of requirements generation and issues capture As projects are tendered to different design teams within different companies at different phases of the project lifecycle it is important to have a record of the HF requirements generated at the early stages so that they can be effectively managed and fulfilled throughout the project. The same is true of HF Issues. It may be that issues have not been closed out in one design phase and will require further investigation by another HF team at a later date. The second HF team will therefore require a clear audit trail in the reporting of the previous team. The requirements and issues close out then becomes a measure of the effectiveness of HFI at the end of the project.
Case studies It is less easy to demonstrate HF benefit at the early stages of the project where the user requirements can be communicated early and drive the design. However, at later stages, there can often be significant design changes required as a result of structured HF and operations requirements generation. The HF consultant is therefore in the difficult position of advocating significant change at the late stages of the project rather than having the opportunity to effectively integrate HF and Operational requirements early in the process. The following case studies illustrate the problems faced in the Integration of HF into Railway Station Upgrade Projects.
Case Study One: Systems integration team present Case Study One illustrates the case of effective systems integration facilitating the dissemination and implementation of HF requirements. The HF team joined the project at the detailed design phase, which typically would be problematic as the detailed design phase begins with a relatively fixed layout in terms of room sizes and adjacencies. However, the architectural and
The effective management of user requirements & human factors issues
519
systems design teams had held early consultation workshops with the client and had incorporated the LU Station Planning standard requirements in the concept design phase (Reference 3). The concept design phase maps out the space and adjacency requirements for the rooms in the station. LU Station Planning requirements specify minimum room size and adjacency requirements and this was followed to the letter. It should be noted that the station was being dramatically altered and therefore that the designers did not have to make compromises on the sizes and adjacencies quoted in the LU Station Planning Standard. These high level user requirements, ascertained from discussions with operators and the ability to follow the station planning standard exactly, provided a baseline structural layout which was found to be eminently suitable for the detailed design of the workstations and equipment within each space. The HF team then reviewed the proposed concept design to determine whether detailed user requirements for in-room layouts of the control facility and ticket office could be accommodated. The systems integration team had been effectively integrating the work of all disciplines prior to HF involvement and it was found that the detailed user requirements were integrated quickly and easily as the forum for integration was already in place. The HF team in this project was seen as the human element in systems engineering which, in the experience of the authors, is an ideal location for the placement of HF input.
Case Study Two: GRIP 5 (detailed design) and no prior HF input Case Study Two illustrates the reverse scenario to the previous case study where end user requirements were not embedded at the early design stages. In this case, it was discovered that during the user requirements capture phase, at the beginning of GRIP 5 (detailed design) the rooms provided for staff accommodation did not fulfil user needs or expectation in terms of room function, numbers of rooms provided, or adjacencies. A high level user requirements document was produced by the HF team based on discussions with the operator which provided the design team with a quick fix room schedule. The room schedule provided high level user requirements for each room including a floor space estimation and adjacency requirements based on equipment and staffing information gleaned from baseline studies, user workshops, task analysis and standards review. The room schedule was colour coded to demonstrate where rooms were not provided in the current design, where the rooms provided were not optimally located or sufficiently sized, and where rooms provided were not now required. This pragmatic approach was followed to reduce time taken to provide input and to provide the project management team with a visual device to determine a way forward as quickly as possible after initial consultations had determined that there may be a significant departure from the original plans. As the room schedule highlighted the magnitude of the change from current to user required layout, the client and project team then had to make the difficult
520
C. Munro & S.R. Layton
decision of whether to incorporate the changes; balancing design time delay and revised design costs against the long term operational costs. The client determined that the future operational costs would be too great and instructed a change in the baseline design, despite programme delay and additional build costs. This case study therefore demonstrates the benefit of user input at early stages of the design to prevent expensive redesign at a later stage as well as the importance of directed communications which help the project management team to reach critical decisions quickly.
Case Study Three: The importance of an operational concept as a basis for user requirements capture Another key factor in successful HF integration is knowledge of the operational processes to be performed in the area. This case study examines a situation where neither operational concept documentation nor access to operational personnel was available to the design team in the early phases of design. The HF team began work late in the detailed design phase but this time without being provided with a clear idea of how the system should function in order to support operations. The project client had specified a prescriptive set of requirements to the design team with no operational explanation to fill in the gaps in information required to produce a design which was fit for purpose. A generic requirements set from a previous similar project was therefore applied to the design to determine where there were significant departures from previous system designs. These significant departures were then flagged to the client’s operations team for consideration. It was then determined in liaison with the client’s operations and systems engineering teams that operational concepts should be produced for the various operations to be performed within the system in order to bridge the gap between prescriptive requirements and operational information. The HF team therefore held several workshops with the client’s operations team to discuss the design and to determine where the design may impose constraints on operations. The issues log and requirements capture tools were especially helpful in this scenario to keep track of the hundreds of issues and thousands of additional design requirements generated by the production of eleven operational concepts.
Discussion The costs and time involved to integrate HF in each of the case studies has varied greatly. Case study one demonstrated an ideal scenario where high level user requirements were incorporated into the wall layout and room adjacencies proposed at concept design and more detailed user requirements generated in the detailed design phase could be accommodated within the spaces allocated. Case study two presented a 3 month project delay and associated costs of all discipline redesign.
The effective management of user requirements & human factors issues
521
In case study three, a greater number of iterations to the design were required to incorporate late changes to the operating baseline and therefore the user requirements set. It is estimated that lack of operational information in the early phases and late change requests have increased the detailed design costs by approximately 20%. The issues and requirements capture process described has been found to be invaluable in managing and communicating human factors, and to provide a clear audit trail of design decisions to support the final HF assurance process. From experience of these and other similar projects, the authors conclude that effective integration of HF is dependant on a number of factors which contribute to the effectiveness of the requirements and issues capture process: • Availability of operational information; • Production of a robust set of user requirements based on an understanding of users tasks; • User requirements integration into the concept design phase where room sizes and adjacencies are determined; • User requirements integration into the detailed design phase; • Structured user review of plans at key stages; • Design team integration; • Systems engineering involvement. It is important that HF team is cognisant of these factors when defining the HF input required at each phase and when discussing the implications of integrating HF late in the design process.
References London Underground, 2006. Station Planning. 1-371 A02. London Underground, 2007. Integration of Human Factors into Systems Development. 1-217 A01. Network Rail, 2003. Incorporating Ergonomics Within Engineering Design: Requirements. RT/E/P/24020, Issue 01.
WORKING PEOPLE
PREDICTING THE HUMAN RESPONSE TO AN EMERGENCY G. Lawson1 , S. Sharples1 , S. Cobb1 & D. Clarke2 1
Human Factors Research Group, University of Nottingham, UK 2 School of Psychology, University of Nottingham, UK
This paper presents a case study of an approach for predicting the human response to a domestic fire, using a combination of a talkthrough technique (Kirwan and Ainsworth, 1992) and sequential analysis (Bakeman and Gottman, 1986). 20 participants were asked what actions they would take upon hearing a strange noise in their house, which they were later told was a fire. Each act was recorded and the results were compared to previous research in which people involved in real fires had been interviewed (Canter et al., 1980). A significant relationship was found between the frequency (Spearman’s rho: 0.694, p < 0.01) and sequence (Spearman’s rho: 0.441, p < 0.05) of acts in this study and those from the interviews with people involved in real fires. More work is needed to develop the approach, but this case study indicates that it might have use as a low-cost method which can be used to predict behaviour in an emergency.
Introduction There is little guidance for ergonomists attempting to predict the human behavioural response to an emergency such as a building fire or an industrial explosion. Predicting behaviour in an emergency has several useful applications, for example: populating simulation tools which can be used for training emergency responders; investigating alternative factory or building layouts for emergency evacuation planning; or planning for crowd safety and the response to major emergencies. Basing predictions on previous events is not always possible as gaining access to data such as incident reports can be difficult, and the behaviours exhibited may be specific to the circumstances in which the event occurred. Laboratory studies are also rightly constrained by ethics considerations which prevent putting participants in a dangerous or distressing situation. Laughery (2005) recognises the importance of predicting human performance, and has provided excellent guidance for giving quantitative estimates to support simulating and modelling human performance. He gives examples of a predictive comparison of interaction methods and of predicting the workload demands for driving a car. However, his methods are not obviously applicable to understanding what actions people will take when placed in an emergency, as they require some knowledge of the tasks people will perform or models of human performance before 525
526
G. Lawson et al.
they can be used. Nevertheless, they may be useful when attempting to quantify or analyse behaviours identified through other methods. Attempts to predict human behaviour are reported in computer simulation literature, as developers of software tools use these predictions to programme computer representations of people to increase the realism of their simulations. An approach used by several developers of simulation models is to research published models of human behaviour, then implement these as code in their simulation tools (Cornwell et al., 2002, Silverman et al., 2006, Pan et al., 2006, Baines et al., 2005). However, this approach has been described as difficult for the following reasons: published models of behaviour can be unspecific, un-quantified or incomplete; insufficient integration exists between different areas of research; developers have insufficient knowledge and understanding of psychology and behaviour; and poor communications exist between people working in social sciences and computer programming (Silverman et al., 2001, Silverman et al., 2006). This study is an initial investigation towards a new approach for behaviour prediction. The approach draws from existing methods used for studying and analysing behaviour. Sequential analysis (Bakeman and Gottman, 1986) is an established method for studying behaviour, which involves the study of events or interactions as they unfold over time, and is therefore particularly suited to studying dynamic aspects of behaviour. The basic method is to record behavioural sequences using a taxonomy of behaviours. For example, if a person displays behaviour X, this is recorded. The type of behaviour displayed subsequently (e.g. Y) is also recorded, and so on for a sample of people, for any task/scenario of interest. This data can be used to identify recurrent sequences of behaviours, and the probability of one particular type of behaviour following another (e.g. the probability that behaviour Y will follow behaviour X). If the results of the analysis are represented in diagrammatical format they can demonstrate the flow of different behaviours and the probability of moving from one to another. Furthermore, generic behaviours can then be abstracted from the analysis. This technique has already been applied to investigate patterns of behaviour in domestic, multiple occupancy and hospital fires (Canter et al., 1980). A team of researchers recognised that fire safety regulations were based upon invalidated assumptions of behaviour in a fire and therefore attempted to gain more detailed and empirical data. Initially, they obtained information from the fire brigade regarding the occurrence of different types of fires. They then attempted to gain statements from witnesses involved in the fires that had occurred, supplementing this with information from press reports and police witness statements. The witnesses were asked to describe exactly what they did from the time they noticed something abnormal was happening, until after they exited the building. These descriptions were transcribed and were coded against a taxonomy of behaviours. The researchers then recorded the frequency with which each of the behaviours followed every other behaviour. From this they generated decomposition diagrams demonstrating the probability of moving to any one of the behaviours from any other stage in the sequence. Finally, they attempted to extract generic models of behaviour which were displayed in all types of residency studied. Thus, sequential analysis was used as a method for modelling and predicting behaviour in emergency situations.
Predicting the human response to an emergency
527
The approach presented in this paper aims to yield the same type of sequential analysis as in the work by Canter et al. (1980), but using individuals’ predictions of their likely behaviour in a fire rather than actual data from those who have experienced fires. However, sequential analysis must be applied to observable or recordable behaviours; as a technique it only describes and analyses behaviour, it does not generate any behavioural phenomena. For this study, a talk-through method was selected as an appropriate method of generating the behaviours of interest. In this method, respondents are simply asked to describe their actions in response to a scenario or statement. It is a low-cost approach, as no special equipment is required (Kirwan and Ainsworth, 1992). Furthermore, using the talk-through technique meant that a dangerous situation could be described and participants’ responses, feelings and opinions could be recorded without putting them in actual danger. As this method is low cost and quick to implement, it was chosen for this study in lieu of observable or reportable behaviour in a real fire.
Method The method was selected such that the study described above (Canter et al., 1980) could be used to validate the results from this case study. 20 participants (10 male; 10 female; mean age = 36, range = 20–52) were recruited from staff and students of the University of Nottingham. Participants were allocated a one hour appointment. They were asked to sketch a plan layout of their house, and identify who would typically be in their house at night-time. Participants were then asked to detail, in order, every action they would take after hearing a faint cracking noise coming from their kitchen, which on investigation they were told was a fire. They were told to stop their description when they reached the point where they imagined they would be completely out of danger, typically upon exiting their house. If their responses contained insufficient detail, or if they missed a logical stage, they were probed with statements such as “I think you’re missing something there” or “I’d like more detail about the stages between those acts”. Their stated acts were recorded using a laptop computer, which the participants were able to check as it was projected onto a display screen. The sketches of the participants’ plan views of their houses were also recorded. The whole experiment was conducted with procedures which received approval from the University of Nottingham Faculty of Engineering Ethics Committee. The example used was informed by a number of assumptions. First, it was assumed that the fire started in the kitchen as anecdotal evidence suggests that this is where most fires begin (e.g. Bosley, 2003). Participants were only told they could hear the smoke alarm upon entering the kitchen. This second assumption was unsupported by data from real fires, but prevented participants from stating they would exit the property as soon as they heard a fire alarm, a sequence of events which was experienced in the pilot studies but was not supported by Canter et al. (1980), probably due to an increase in the prevalence of smoke alarms since the original study. If participants stated that they would fight the fire, they were told that despite their attempts, the fire would not be put out. Also, if participants reported
528
G. Lawson et al.
that they would send someone else to investigate the noise, they were told that this person informed them that there was a fire upon their return. The actions described by the participants were coded using the taxonomy of acts generated by Canter et al. (1980). Where participants described an act which could not reliably be mapped to an act in the original study, new codes and activities were created. The coding of participants’ behaviours was reviewed by an independent researcher to check for consistency. A matrix was then created in which the number of times each act followed every other act was recorded. The matrix was used to generate standard residuals for each transition (observed frequency minus expected frequency, divided by square root of expected frequency). Canter et al. (1980) reported these values as “strength of association” and used them to provide information on the relationships between acts on decomposition diagrams (see Figure 2).
Results Initially, the 49 acts from the original Canter et al. (1980) study were scrutinised, and any which would not have been expected in this case study due to the example scenario were omitted from the analysis. These were mainly related to actions for which insufficient detail was reported in the original study, and therefore to use them in this study would have introduced inaccuracies. For example: any tasks regarding involvement with smoke were omitted as there was no data on the spread of smoke in Canter et al. (1980); any act in response to another person, aside from a partner returning after investigating the fire, was omitted as again no data was published in the original study on the actions of the other people; struggling with fire equipment and any action relating to someone who was not from the house containing the fire were also omitted. After this process, 23 of the 49 acts originally described by Canter et al. (1980) were deemed suitable for comparison. This corresponded to a total of 233 statements (or 65%) of all 361 statements reported in this experiment; 34 statements (9%) were excluded as a result of the process described above, the other 94 statements (26%) were reported in this experiment but could not be mapped onto acts in the original taxonomy. A scatter plot of the results, showing the frequency of comparable acts from the original study against those from this experiment is shown in Figure 1. A Spearman’s rho test demonstrated that a significant relationship existed between the frequency of acts in the original study which were compared to those from this experiment (Spearman’s rho: 0.694, p < 0.01). The transition matrix was used to investigate the sequence of actions. The standard residuals were identified for any sequences between the 23 comparable acts for which values were reported in Canter et al. (1980). For these transitions, the correlation between the standard residuals reported in Canter et al. (1980) and from this study was found to be significant (Spearman’s rho: 0.441, p < 0.05). The values used in the comparison are shown on a decomposition diagram in Figure 2. The strength of association values from Canter et al. (1980) are labelled on arrows which point to subsequent acts; the corresponding values from this experiment are shown in brackets. There is no meaning in the position or proximity of the nodes.
Predicting the human response to an emergency
529
80 70
Canter et al. (1980) results
60 50 40 30 20 10 0 0
Figure 1.
5
10
15 20 Experiment results
25
30
35
Comparison of the frequency of 23 acts from Canter et al. (1980) to corresponding acts from this experiment.
Discussion In this instance, the technique demonstrated a statistically significant relationship between the frequencies of acts found in this study with those from the original interviews with survivors of real fires. The sequences of the acts also demonstrated a statistically significant relationship with those from the original study. Therefore, in this example, a predictive approach was able to provide an indication of how people will behave in a domestic fire. It was achieved with minimal resources; 20 people each giving up to one hour in a basic office meeting room with a laptop, projector and whiteboard. There are aspects of the predictive approach which require further development. In particular, the validity needs to be assessed. Although face validity of this approach appears to be low – sitting comfortably in an office environment during the day is very different to be being woken by a fire in a home at night – the approach did produce a reasonable prediction of what actions people will take, when measured against the behaviour of people in actual domestic fires. However, this study only demonstrates comparability with another study. If access was gained to a more detailed incident report from an emergency scenario, this would allow for a more fine-grained analysis of the behaviours, and would also permit inclusion of some of the acts omitted from this study, such as interaction with smoke. A more detailed account of human behaviour in fire would also allow inclusion of timing of acts, as well as frequencies and sequences.
530
G. Lawson et al. Wait for person to return 2.68 (0)
Informed (discuss)
1.34 (⫺1.10) 0.55 (⫺1.05) Investigate 2.46 0.97 (2.09) (⫺0.22) Dress Encounter 1.79 Instruct/ 1.72 smoke/fire (1.12) 3.13 reassure (0.69) 2.12 (3.11) (⫺0.02) 8.88 0.45 Enter room (6.87) Feel (1.08) of fire origin 1.34 concern Fight fire 0 (⫺0.24) 3.58 (1.99) Go to neighbours 0.89 (7.49) (4.87) or return to house 2.24 (5.83) Close door 0.45 (0.87) 4.47 Evasive (2.01)
Warn (phone fire brigade) 0.89 0.89 (⫺0.22) (0)
14.3 Leave house (2.62) Meet fire brigade on arrival 4.47 4.47 (⫺0.90) (10.06)
Figure 2. Decomposition diagram showing strength of association values from Canter et al. (1980), and those from this experiment in brackets.
The generalisability and reliability of the technique cannot be evaluated based on this case study alone: it would need to be repeated and applied to other types of emergency situations. The Canter et al. (1980) study provides results from multiple occupancy and hospital fires for which this technique could be applied to investigate the generalisability. Work is already underway to begin investigating the reliability by repeating this study with a further 20 participants. Another limitation of the study is that performance influencing factors such as time-of-day effects, fatigue and training have not been accounted for. It is acknowledged that these are likely to influence behaviour, particularly in an industrial setting. However, the study used for comparison of results (Canter et al., 1980) had no information regarding these factors. Therefore to include them in this study would have introduced confounding variables. They should, however, be investigated as part of the future development of this approach. The approach only reveals what actions the respondents predict they would do; it does not reveal the causes of their behaviours. That is, it attempts to answer the question “what would people do?” not “why would they do it?” Obviously the
Predicting the human response to an emergency
531
latter would be of interest to an ergonomist when attempting to fully understand the behaviours, and how they might be influenced by workplace design or training. Despite the limitations and further work required, in this example the approach was able to predict many of the behaviours people may demonstrate in a domestic fire, and gave an indication of their sequences. Although it does not identify or provide an understanding of the reasons why people exhibited these behaviours, it was a low-cost and efficient method, which did not rely on experts or people who have experienced a fire. Therefore, it is anticipated that it could have use as a “first-pass” method for predicting behaviour in an emergency, which could then be supplemented by other more involved approaches. For example, the results from this study could be used to increase the realism of the computer representations of people in a simulation tool by incorporating probabilities of various behaviours. The simulation could then be reviewed and further refined through other methods to increase its validity.
Conclusions This case study examined the use of a new approach to predicting human behaviour in a domestic fire. The technique was low cost and was conducted quickly, yet in this example still gave reasonable results for the frequency and sequences of acts when compared to another study based on past events. Therefore, it might have use as a method for predicting behaviour in new situations for which there is no existing knowledge. However, further work is required on the method, in particular to test reliability and generalisability and to develop greater confidence in the validity of the results. With development, there is potential for this approach to offer a variety of uses for understanding how people will behave in an emergency situation. For example, in an industrial setting, this approach could be used to generate information that could help develop appropriate response plans, evacuation procedures, signage, training programmes or building layouts. It is anticipated that despite the short-comings, the approach could be used to quickly obtain indications of how people might behave, with minimal resources.
Acknowledgements The authors acknowledge Professor David Canter and John Wiley & Sons Ltd for permission to use and modify material from Domestic, Multiple Occupancy, and Hospital Fires (Canter et al., 1980).
References Baines, T. S., Asch, R., Hadfield, L., Mason, J. P., Fletcher, S. and Kay, J. M. 2005, Towards a theoretical framework for human performance modelling within manufacturing systems design. Simulation modelling practice and theory, 13, 486–504
532
G. Lawson et al.
Bakeman, R. and Gottman, J. M. 1986, Observing Interaction: An Introduction to Sequential Analysis, (Cambridge University Press, Cambridge) Bosley, K. 2003, Domestic Firefighting. Accessed on World Wide Web January 2008. Available from: http://www.rmd.communities.gov.uk/project.asp? intProjectID = 10887. Canter, D., Breaux, J. and Sime, J. 1980, Domestic, Multiple Occupancy, and Hospital Fires. In D. Canter (ed.) Fires and Human Behaviour. (John Wiley and Sons, New York). Cornwell, J. B., Silverman, B. G., O’Brien, K. and Johns, M. 2002, A Demonstration of the PMF-Extraction Approach: Modeling The Effects of Sound on Crowd Behavior. In Proceedings of the Eleventh Conference on Computer Generated Forces and Behavioral Representation. (Orlando). Kirwan, B. and Ainsworth, L. K. 1992. A Guide to Task Analysis (Taylor and Francis, London) Laughery, R. 2005, Simulation and modelling for analysing human performance. In J.R. Wilson and N. Corlett (eds.) Evaluation of Human Work, Third Edition, (Taylor and Francis, London). Pan, X., Han, C. S., Dauber, K. and Law, K. H. 2006, Human and social behavior in computational modeling and analysis of egress, Automation in Construction, 15, 448–461 Silverman, B. G., Johns, M., Cornwell, J. and O’Brien, K. 2006, Human Behaviour Models for Agents in Simulators and Games: Part I – Enabling Science with PMFserv, Presence: teleoperators and virtual environments, 15, 139–162 Silverman, B. G., Might, R., Dubois, R., Shin, H., Johns, M. and Weaver, R. 2001, Toward a Human Behaviour Models Anthology for Synthetic Agent Development. In 10th CGF Proceedings, (IEEE, New York)
HUMAN FACTORS CONTRIBUTIONS TO THE DESIGN OF A SECURITY PROMOTION CAMPAIGN: A CASE STUDY A.E. Peron, B.F. Davies & S.R. Layton Human Engineering Limited, Bristol, UK Human Engineering Limited was recently involved in the design of a security promotion campaign at London Luton Airport. The aim of this campaign was to further enhance adherence to Access Control procedures by staff at LLA by optimising security awareness, through the provision of a persuasive message designed to change attitudes and subsequently behaviour. This work represented a short turn-around project, based on the development of series of broadranging recommendations in terms of promotion, message content, and medium and design of delivery. The recommendations provided represent a good example of an integrated human factors approach in a challenging work environment and were well-received by the airport staff, Airport Operators at the AOA Safety and Operations Conference, and the Department for Transport.
Introduction This paper outlines the background to a security promotion campaign at London Luton Airport (LLA), which was based on recommendations made by Human Engineering Limited (Human Engineering). These include the design of message mediums for the security promotion campaign for Access Control, and for adherence to Body Search procedures. The aim of the campaign was to further enhance adherence to Access Control procedures by staff at LLA, through the provision of a strong message designed to change attitudes and subsequently behaviour. Overall, this campaign aimed to engender a culture of ID pass awareness, of ownership of the airport and of importance placed on procedure. This paper draws attention to the important role of psychological factors in security in restricted areas in airport and other locations, which can explain why staff who have the authority and duty to maintain Access Control may not do so. In 2007, London Luton Airport was judged to be the fifth largest airport in the United Kingdom (UK), and one of the fastest-growing major UK airports with 9.9 million passengers, which represented an increase of nearly 400% in 10 years. In its continued commitment to maintain high levels of security, LLA is keen to optimise security awareness across all members of staff who work at the airport, not just those working in the Security Department. Two major security issues identified by 533
534
A.E. Peron, B.F. Davies & S.R. Layton
the Operations Director at LLA that could be addressed in this endeavour were; Access Control and Body Search. The underlying reasons identified by the Operations Director of LLA for the instances of non-optimisation of these issues include: • Lack of confidence by staff to enforce procedure, despite being knowledgeable about it (e.g. ID passes – allowing unauthorised access or no pass checking, failures in the correct display of ID passes by some staff) • Lack of knowledge of airport/procedures (e.g. inadequate understanding of zones) • Lack of awareness of security as a universal responsibility across airport staff • Lack of strict adherence to the procedure – able to be ‘fobbed off.’ LLA proposed implementing a security promotion campaign, to address these issues. It was determined that Human Engineering could provide LLA with support to the security promotion campaign through assistance in designing promotional messages aimed to improve Access Control. It was also determined that, while the Body Search issue would require further work, certain psychological recommendations could be made to facilitate the procedure. However, these recommendations will not be reported herein. Approximately 300 security staff are employed by London LutonAirport Limited, with a further 48 employed by contracted service providers, and a similar number of in-house staff at Signature Flight Support. Approximately 10,000 individuals, including baggage handing staff, check-in staff, cleaners, maintenance, etc., are authorised access to different zones of LLA, with some 2,500 of these authorised to access the secure airside area. The airport is broadly divided into two security areas: the Restricted Zone (RZ), which represents the commercial passenger traffic and holds the highest level of security; and the Control Zone (CZ), which represents cargo and private planes, and holds a lower level of security. Within these two security areas, the airport is divided into 8 zones, as outlined by the National Aviation Security Programme. These are: Internal areas of the RZ; Baggage Reclaim Halls; Baggage Make-Up Area; Ramp; Aircraft; Other areas of the RZ; Landside; All areas. According to role, each LLA authorised-individual may access a different number of these 8 zones. The individuals’ picture ID pass clearly indicates the name, job title, and zones that they are allowed to access. The ID pass is to be worn at chest-height or in an easily-visible alternative location if their chest area is often obstructed (such as upper-arm). A recent Department for Transport (DfT) inspection commented that more than 90% of staff at the airport were carrying their ID passes appropriately. Access Control, or the control and monitoring of individuals across zones, is a critical issue for security. Security staff are well-trained in access control via control points. All staff are made aware when they receive their ID passes that controlling zone access is an important issue, and if they are unable to see an ID pass on another individual, they should check the zone access information on that pass. If the individual is not authorised, they should be escorted out of the zone and security should be alerted.
Human factors contributions to design of security promotion campaign
535
Method Human Engineering conducted a series of data collection activities to establish key factors underlying non-optimisation of security procedures at LLA. First, naturalistic observations at LLA, both airside and landside, were conducted by two experienced consultants, over the course of a day. These observations enabled Human Engineering to observe operational procedures at the airport, and to note failures to act in accordance with security procedures. Areas in which these observations were conducted included the Central Security Zone (CSZ), Baggage Hold, and all points where staff could pass from land to airside. Initially, staff were blind to the observations, to minimise demand characteristics. To actively assess staff responses to security violations, Human Engineering conducted a series of activities which Airport Operations management staff judged to represent possible security risks. These included entering secure zones without proper identification and spending time in the baggage hold pretending to tamper with baggage, all in the absence of any senior airport management staff (to ensure staff did not feel the activities were authorised due to the presence of management). In addition, Human Engineering observed a DfT mule1 pass through two automatic metal detectors (AMDs) in the CSZ. Finally, Human Engineering conducted a series of semi-structured interviews with open-ended questions with a number of staff (approximately 10 participants) across the airport, to gain an understanding of the factors underlying non-optimisation of security procedures and their opinions of security at the airport. An informal but structured thematic analysis was conducted on the content to elicit significant points, particularly relating to causes for the nonoptimisation of security procedures. The results of these activities were discussed in the project team to determine the appropriate context and nature of the promotion campaign, and a number of recommendations for the design of the campaign were developed. For all security promotion campaign materials recommended in this paper, style guidelines and design suggestions were provided in the original report presented to LLA. Following design guidance, LLA implemented a campaign based on these recommendations and the underlying philosophy as provided in the report.
Discussion The results of the data collection activities revealed that there were instances of non-optimisation of security procedures, and that staff did not feel motivated to execute security procedures at all times, and did not feel personally responsible for airport-wide security. The themes to explain the non-optimisation of security procedures which emerged from the observations and interviews conducted
1 An actor simulating a person intentionally or unwittingly carrying contraband through, or into, a restricted zone.
536
A.E. Peron, B.F. Davies & S.R. Layton
were consistent with those reasons identified by the Operations Director of LLA, namely: • Lack of confidence by staff to enforce procedure, despite being knowledgeable about it (e.g. ID passes – allowing unauthorised access or no pass checking, failures in the correct display of ID passes by some staff) • Lack of knowledge of the airport/procedures (e.g. inadequate understanding of zones) • Lack of awareness of security as a universal responsibility across airport staff • Lack of strict adherence to the procedure – able to be ‘fobbed off.’ The results were discussed in the project team to determine the appropriate context and nature of the promotion campaign, and a number of recommendations for the design of the campaign were developed, and have subsequently been adopted by LLA.
Recommendations for a promotion campaign to enhance Access Control Instances of non-optimisation of Access Control in LLA represent failures of individuals to act in full accordance with security procedures. The best predictor of action is the strength of an individual’s intention to act. The Theory of Planned Behaviour (TPB; Ajzen, 1991) states that in order to improve the intention to act, the perceived behavioural control of an individual must be increased. In other words, intention to act is a function of the belief that we have the cognitive resources necessary to enact the behaviour. In this instance, resources relate to: • Confidence in the knowledge of the procedure; • Confidence in systemic support; • Confidence in own judgement. Individuals who are confident that they are aware of the rules, are confident that their judgements are correct, and are confident that the system will support them in their judgements, will be more likely to act. It was determined that the message provided by a security promotion campaign for Access Control must target the improvement of these resources, through the use of persuasive communication. The object of the message, in this case staff at LLA, are termed the ‘recipients’ of the message, and will be referred to henceforth as such. A persuasive message is one that is more likely to result in a change in attitude and behaviour, as it leads to more in-depth, elaborative cognitive processing (Chaiken, 1987; Petty and Cacioppo, 1986). Criteria for a persuasive message are summarised in a table in the original report presented to LLA, and it was noted that all of the material produced for this campaign by LLA should be compared against these guidelines. In this instance, it was determined that there was one overall message (recipients should follow security procedures for Access Control) and sub-messages to target confidence (in knowledge, systemic support, and individuals’ judgments). The overall sub-messages within the campaign were categorised into two types; Informational (to do with procedure) and Educational (teaching the recipient to avoid
Human factors contributions to design of security promotion campaign
537
psychological traps). The informational sub-message was designed to address the confidence in the knowledge of the procedure and in systemic support, and the educational sub-message was designed to address the confidence of individuals in their own judgements. It was determined that the key messages of the security promotion campaign were best captured in the form of slogans, or memorable mottos or phrases. Condensing the message reduces the cognitive demand on the recipient to recall it. Generally, a campaign is presented via a primary slogan, which is generally the name of the campaign. Including one slogan across all sub-campaigns within the message helps maintain cohesion across the overall campaign. In this instance, the slogan ‘Act on Access!’ was proposed, which encapsulated this broad message in a memorable, low-cognitively demanding way, and the alliteration of ‘Ac’ also rendered it more memorable. In addition to a primary campaign slogan, a secondary slogan to present a different aspect of the overall message is useful. This enables more information to be presented to the recipient, in a way that will be easily recalled. Generally, a secondary slogan should provide information about how to achieve the overall recommendation, i.e. information on procedure. In this case, ‘Check and Challenge!’ was suggested as the secondary slogan in this campaign. ‘Check and Challenge’ was specifically developed to reinforce the procedural aspect of the task, i.e. it is a rule-based, staged approach (first check, then challenge if appropriate). In addition, using the word ‘Check’ as the first step of the procedure was thought to minimise the confrontational aspect of Access Control, which may have caused concern to some. Reinforcing the first stage as a non-confrontational procedure was judged to remove the emotional-load of the task, which may have presented a psychological barrier to some recipients to act. The more emotionally-loaded, confrontation term of ‘Challenge’ was reserved for the second stage of the procedure. It was concluded that generally, the vast majority of the time, recipients only need to check, and will not progress to the challenge stage. It was agreed that the message would be presented to recipients via a series of posters, which were displayed at multiple locations airside in LLA, and a creditcard sized (85mm × 55mm) Act on Access card. These mediums were agreed to be easily accessible: the posters were highly visible, and carrying the Act on Access card was incorporated into the existing procedure of carrying the ID pass (it should slot behind the ID pass). This was judged to increase the confidence of the recipient to refer to the Act on Access card, as it was paired with their badge of authority, which reinforced their own authority and thus their confidence to act. Both mediums relied on pictorial representation of the message where possible, to accommodate staff for whom English is not a first language. Where pictorial representation was not possible, the complexity of language used was kept very simple, again to accommodate those with English as a second language. It was suggested that the posters in this campaign presented both the informational, and educational sub-messages of the campaign. Informational posters are designed to support the recipients’ knowledge of the procedure. They aim to increase the confidence of the recipient in their knowledge of the procedure, and their confidence in systemic support coming from management. It was concluded that ultimately, the information provided in the informational posters would increase
538
A.E. Peron, B.F. Davies & S.R. Layton
Figure 1.
Poster indicating zone.
Figure 2.
Poster indicating zone.
the third resource (confidence in judgement), as the recipient had clear evidence to support their judgement. Three posters were recommended for this series, focusing on: • Information (clarification) of zones • Information (clarification) of authorised cards within LLA • Information (clarification) on the Access Control procedure. In terms of the need to clarify zone information, it was suggested that this was done in a combination of 3 ways: 1. Poster of the existing LLA zone map. This should be placed no further than a 10 second walk from any point, unless option 3 is adopted in which case they can be placed a greater distances, i.e. one in each zone 2. Modified version of the existing LLA zone map, to be location-specific (mark X at location of the poster). This should be placed no further than a 10 second walk from any point, option 3 is adopted in which case they can be placed a greater distances, i.e. one in each zone 3. Small posters with zone number (ideally colour-coded by zone) placed such that this information is visible at a distance from any point (see Figures 1 and 2). The ideal recommended combination for zone information display was a combination of options 2 and 3. The educational posters proposed for this campaign were designed to improve the judgement of the recipient, for example, judging when to exercise the authority to Check and Challenge. Fundamental psychological principles can explain why staff may have failed to Check and Challenge, or why, when they did, they failed to execute the full procedure. The expertise heuristic (Hass, 1981, see also Eagly et al., 1978; Hovland & Weiss, 1951; Maddux and Rogers, 1980) and the attractiveness heuristic (Chaiken, 1979, see also Chaiken, 1986; Dion et al, 1972; Hatfield & Sprecher, 1986) are two cognitive phenomena that can account for these failures. Due to the limitations of human cognition, humans exert cognitive effort judiciously. Cognitive short-cuts, or heuristics, enable the conservation of resources where possible. Allison et al. (1990) explain a heuristic as a cognitive process which requires little cognitive ability and motivation. An example of a heuristic is a stereotype, where additional information is provided to the individual at little cost,
Human factors contributions to design of security promotion campaign
539
as elaborate assumptions or judgements about something or someone are made, simply because they appear to belong to a category. Making these assumptions is easier than actively gathering information to create a new judgement for that situation. The expertise heuristic and the attractiveness heuristic are two powerful cognitive short-cuts that humans use. Simply put, the expert heuristic relates to the common assumption that if someone looks like an expert, they are. This relates directly to authority – if someone looks like they are in a position of authority to be in a certain location, they are. Similarly, humans are favourably drawn towards attractive people. The attractiveness heuristic refers to the effect that if someone is attractive, individuals are predisposed to like them and to trust them, and thus will not thoroughly process the actual message they are giving. Advertisers often rely on these heuristics to achieve their aims. In the airport-environment, the expertise heuristic can easily be seen in the frequently-reported ‘high-visibility effect,’ where donning a high-visibility jacket lends an air of authority that can be seen in the number of people approaching the wearer for assistance. In addition, the expertise heuristic is supported by the strong sense of embarrassment associated with questioning an authority figure, and this may also play a part in preventing airport staff from checking and challenging a stranger’s access rights. Thus, the aim of the educational posters was to teach recipients that what an individual’s appearance, and how they act, is not necessarily an accurate representation of their true position. The educational posters aimed to reinforce the Check and Challenge procedure through demonstrating that it should be applied universally, regardless of the appearance of the individual in question. Four posters were recommended in this series, focusing on both the expertise and attractiveness heuristics. It was recommended that these posters presented human models that represent the expertise and attractiveness heuristics of UK society. It was recommended that a male and female version be used to represent each of these heuristics. This would ensure that the heuristics were kept gender-neutral. It was recommended that the educational posters used emotion, specifically fear, to enhance the persuasive message of the campaign. Fear is a powerful tool that encourages recipients to attend to a message, and act on it to avoid the adverse consequences exhibited in the message. In this instance, the illustration was a seemingly-harmless model carrying a dangerous weapon (e.g. a gun or a bomb, as these were judged to be prime concerns by the Security Manager at LLA). In addition to the posters, it was suggested that all staff were presented with an ‘Act on Access’ card, a credit-card size card that can be slotted in behind the recipient’s ID pass. This card was intended to serve as an aide memoire and reference tool, for individuals to confirm their knowledge, and also to provide confidence to the individual in the procedure and how to enforce it. Both sides of the card were used. One side of the card contained a summary of the procedure of how to Check and Challenge. On the other side, the space was divided in half vertically: on one half, there was a summary of zone information, and on the other half, a list of tips on how to execute the Check and Challenge procedure. This list was designed to provide support to individuals who are inherently more nervous about approaching others, and to provide them with tips to increase their confidence to do so. This card was also designed to reinforce the message that the recipient would
540
A.E. Peron, B.F. Davies & S.R. Layton
never be penalised for following the Check and Challenge procedure. In addition, to ensure the success of the Act on Access! campaign, it was recommended that LLA encouraged staff to keep existing ID passes in good and current condition, such that all information contained therein was legible, and that the ID picture was a current and accurate representation of the pass holder. Regular updates to the ID pass photo were recommended. Subsequent staff interviews reveal popular opinion that the changes made at LLA to date represent a very positive step forward. Enhanced levels of performance in security protocols are being achieved (i.e. improved detection rates of mulecarried weapons). Senior management at LLA report that the security promotion campaign has resulted in enhanced security awareness and confidence by staff at LLA, which has been noted by the DfT to the extent that they are promoting the LLA model as ‘best practice’ for security promotion and enhancement at other airports in the UK. As with any project based on short-term observations, there are limitations to the recommendations and best practice guidance presented to LLA, summarised in this discussion. In general, the recommendations presented were based on very limited observation and the sample of representative end-users interviewed was relatively small. In addition, a full and comprehensive qualitative analysis of the interview data was not conducted, due to time pressure. The recommendations presented relating to the Act on Access! campaign assumed that ID pass security was adequate, and that airport staff were trained about the importance of keeping their passes safe and reporting their loss immediately. The campaign assumed that if an individual had a pass with a picture that resembles the carrier of that pass, they were authorised in that zone. It was not designed to promote awareness of ID pass theft. Finally, the project budget did not allow for the determination of key performance indicators, by which any enhancements in security procedures as a result of the campaign could be measured quantitatively. In the future, it is imperative that this and similar campaigns are supported with staff education and training.
References Allison, S., Worth, L. and Campbell King, M. 1990, Group decisions as a social inference heuristic. Journal of Personality and Social Psychology, 58, 801–811. Azjen, I. 1991, The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179–211. Chaiken, S. 1979, Communicator physical attractiveness and persuasion. Journal of Personality and Social Psychology, 37, 1387–1397. Chaiken, S. 1986, Physical appearance and social influence. In C.P Herman, M.P. Zanna, and E.T. Higgens (eds.) Physical appearance, stigma, and social behaviour: The Ontario Symposium, 3, (Hillsdale, NJ: Erlbaum), 143–177. Chaiken, S. 1987, The heuristic model of persuasion. In M.P. Zanna, J.M. Olson, and C.P. Herman (eds.) Social Influence: The Ontario Symposium, 5, (Hillsdale, NJ: Erlbaum), 3–39.
Human factors contributions to design of security promotion campaign
541
Dion, K., Berscheid, E. and Walster, E. 1972, What is beautiful is good. Journal of Personality and Social Psychology, 24, 285–290. Eagly, A.H., Wood, W. and Chaiken, S. 1978, Causal inferences about communicators and their effect on opinion change. Journal of Personality and Social Psychology, 36, 424–435. Hass, R.G. 1981, Effects of source characteristics on cognitive responses and persuasion. In R.E. Petty, T.M. Ostrom and T.C. Brock (eds.) Cognitive responses in persuasion (Hillsdale, NJ: Erlbaum), 44–72. Hatfield, E. and Sprecher, S. 1986, Mirror, mirror: The importance of looks in everyday life, (New York: SUNY Press). Hovland, C.I. and Weiss, W. 1951, The influence of source credibility on communication effectiveness. Public Opinion Quarterly, 15, 635–650. Maddux, J.E. and Rogers, R.W. 1980, Effects of source expertness, physical attractiveness, and supporting arguments on persuasion: A case of brains over beauty. Journal of Personality and Social Psychology, 39, 235–44. Petty, R.E. and Cacioppo, J.T. 1986, The Elaboration Likelihood Model of persuasion, (New York: Academic Press).
Author index
Ackroyd, P. 145 Adey, P. 453 Anokhin, A.N. 135 Baber, C. 255, 282 Behrendt, S. 405 Benedyk, R. 393 Benger, J. 174 Berman, J. 145 Binch, S. 245 Binnersley, J. 159 Bottomley, D. 245 Braithwaite, R. 332 Buckle, P. 33 Burnett, G.E. 503 Bust, P.D. 432 Caird-Daley, A. 320 Case, K. 51, 61, 70 Clarke, D. 525 Clarkson, P.J. 483 Cobb, S. 525 Cousens, R. 332 Davies, B.F. 533 Davis, P.M. 51, 61, 70 den Besten, O. 453 Devereux, J.J. 189 Drury, C.G. 3 Dul, J. 16 Dunham, P. 462 Evans, J.L. 100 Evans, S. 236 Gale, A. 347 Gkikas, N. 423 Gramopadhye, A. 363, 373 Grandt, M. 282 Gyi, D.E. 51, 61, 70
Hasseldine, B. 442 Henshaw, M.J.D. 100 Hignett, S. 174 Hill, J.R. 423 Hodgson, A. 81 Horton, J. 453 Houghton, R.J. 255, 282
Richardson, J.H. 423 Robertson, S. 432 Rydstedt, L.W. 189
Irune, A. 503 Jakob, M. 405 Jenkins, D.P. 272, 300 Jones, A. 174 Kenwright, M. 491 Kinross, M. 453 Knight, J.F. 255 Kraft, P. 453 Kyriacou, P. 159 Lawson, G. 525 Layton, S.R. 513, 533 Leach, P. 145 Liebers, F. 405 Liu, Y. 182 Lundqvist, P. 414 Marsh, J. 332 Marshall, M. 245 Marshall, R. 51, 61, 70 Mason, M. 332 Molloy, E.M. 91, 100 Mullins, P. 473 Munro, C. 513 Newman, M. 453, 462 Noy, Y.I. 24 Noyes, J.M. 218 Osvalder, A. 182
Hamilton, W.I. 291 Handley, H.A.H. 310 Haslam, R.A. 100
Peron, A.E. 533 Pettitt, M. 39 Pleshakova, N.V. 135 Porter, J.M. 51, 61, 70
Patel, H. 39 Pennie, D. 473, 491
543
Sadasivan, S. 363 Salmon, P.M. 272, 300 Schwaninger, A. 381 Sharples, S. 208, 525 Sherwood Jones, B.M. 229 Siemieniuch, C.E. 81, 91, 100 Sims, R.E. 51, 61, 70 Sinclair, M.A. 91, 100 Sirett, P. 332 Smillie, R.J. 310 Smith, K.T. 263 Stanton, N.A. 272, 300 Stedmon, A.W. 432, 442 Stedmon, D.M. 432 Stone, R.J. 320 Stringfellow, P. 363, 373 Sugden, C. 245 Summerskill, S.J. 51, 61, 70 Taylor, J. 111 Thomas, S. 122 Wales, A.W.J. 381 Walker, G.H. 272, 300 Wallace, L.M. 159 Waterson, P.E. 165 Wilson, J.R. 39 Woodcock, A. 111, 159, 393, 453, 462 Woodcock, K. 354 Wright, M. 473 Young, M.S. 199, 442
