NPfIT was established in 2002 to deliver “twenty-first century IT support” for the NHS, defined as a set of electronic services to provide lifelong health records, direct booking of outpatient appointments and transmission of prescriptions from primary care to pharmacy, all based upon a robust information architecture and secure broadband network [1UK Department of Health. Delivering 21st Century IT: Support for the NHS National Strategic Programme. [electronic document] Available from: http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_4067112.pdf 2002 [cited 2007 October 15];].

There has been sustained and widespread criticism, not always well-informed, of the perceived failures of NPfIT and its overly centralist, secretive and monolithic approach [2Anonymous. Computers and the NHS Private Eye 2006 October 27; , 3British Computer Society Health Informatics Forum. The Way Forward for NHS Health Informatics [electronic document] Available from: http://www.bcs.org/upload/pdf/BCS-HIF-report.pdf 2006 [cited 2009 December 15 ];]. For example, the 2002 Gateway Review by the UK Office of Government Commerce (OGC) into the business justification for what was then called the Integrated Care Records Service (only recently published following requests made under the provisions of the Freedom of Information Act [4Collins T. Government ordered to publish reviews of risky IT projects Computer Weekly [electronic document] Available from: http://www.computerweekly.com/Articles/2009/06/22/236526/government-ordered-to-publishreviews-of-risky-it-projects.htm 2009 June 22; [cited 2009 Nov 3];]) stated that they found “much talk of what the IT programme will achieve, but little recognition of the potential impact of this on current practices, procedures and systems, both technical and organisational” [5UK Office of Government Commerce. National Health Service - Integrated Care Records Services. Gateway 1 - Business Justification [electronic document] Available from: http://www.connectingforhealth.nhs.uk/about/foi/g1icrs0275.pdf 2002 October; [cited 2009 Nov 3];].

Similar themes continue to be echoed [6Peltu M, Eason K, Clegg C. How a sociotechnical approach can help NPfIT deliver better NHS patient care [electronic document] Available from: http://lubswww2.leeds.ac.uk/COSLAC/index.php?id=54 2008 [cited 2009 August 21];]. The evaluation report on the NPfIT Summary Care Record pilot implementation in North West England suggested that the project’s failure to adopt a socio-technical approach was one reason why the system was not embraced by busy clinicians [7Greenhalgh T, Stramer K, Bratan T, et al. Summary care record early adopter programme: An independent evaluation by University College London [electronic document] Available from: http://www.ucl.ac.uk/openlearning/documents/scrie2008.pdf 2008 April 30; [cited 2009 Nov 3];].

Comparable problems have been reported in other major health informatics (HI) implementations [8Hanseth O, Jacucci E, Grisot M, Aanestad M. Reflexive integration in the development and implementation of an Electronic Patient Record system In: Hanseth O, Ciborra C, Eds. Risk complexity and ICT. Cheltenham: Edward Elgar 2007., 9Aarts J, Doorewaard H, Berg M. Understanding implementation: the case of a computerized physician order entry system in a large Dutch university medical center J Am Med Inform Assoc 2004; 11(3): 207-16.], so this issue is by no means unique to the English programme.

The roots of socio-technical thinking can be found in research studies by the Tavistock Institute in the 1950s. For example, investigations in the British coal industry found that autonomous groups of miners had re-invented surprisingly effective innovations in mining methods contrary to “obvious” techniques suggested by increasing mechanization [10Trist E. The evolution of socio-technical systems: a conceptual framework and an action research program Occasional paper no 2 [electronic document] Available from: http://orca.ecn.purdue.edu/~grouper/papers/public/evolution_of_socio_technical_systems.pdf 1981 June; [cited 2009 10 November];]. The socio-technical approach was built upon a philosophy of improving worker satisfaction and quality of life, with the belief that this leads to improved productivity and effectiveness [11Mumford E. The story of socio-technical design: reflections on its successes, failures and potential Inform Syst J 2006; 16(4): 317-42.].

In this paper, the “socio-technical” perspective is understood to refer primarily to the insight that a technical system implementation inevitably affects and is affected by the interdependent social system within which and upon which it operates [12Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care - an interactive sociotechnical analysis J Am Med Inform Assoc 2007; 14(5): 542-9., 13Coiera E. Four rules for the reinvention of health care BMJ 2004; 328(7449): 1197-9.]. This concept has been used to formulate design principles [14Clegg C. Sociotechnical principles for system design Appl Ergon 2000; 31(5): 463-77.] (see Box 1) and methods [15Mumford E. Systems design: ethical tools for ethical change. Basingstoke: Macmillan 1996., 16Mumford E. Redesigning human systems. Hershey, PA:: IRM Press 2003.] for information systems.

This approach requires a more nuanced worldview than the positivist mindset typically associated with reductionist physical science and “hard” technology. It utilizes perspectives such as constructivist or interpretive epistemologies and a social interactionist or negotiated view of reality [17Whetton S. Health Informatics: A socio-technical perspective. Oxford: OUP 2005.] involving complex adaptive systems [18Plsek P. Redesigning healthcare with insights from the science of complex adaptive systems In: Committee on Quality of Health Care in America, Ed Crossing the quality chasm. Washington DC: National Academy Press 2001.]. This paper argues that these factors are particularly crucial in HI and introduces a tool that supports adoption of a socio-technical approach in HI.

The organization and practice of healthcare is one of humanity’s most complex endeavours. Clinical work has been described as mostly “unpredictable and non-routine” [19Greenhalgh T. Implementing the National Programme for IT in the NHS: No easy answers: UK Faculty of Health Informatics 2008.] and “highly contingent, ad-hoc and idiosyncratic” [20Aarts J. On Articulation and Localization - Some Sociotechnical Issues of Design, Implementation, and Evaluation of Knowledge Based Systems In: Lecture Notes in Computer Science. Berlin: Springer 2001; 2101: pp. 16-9.]. Dealing with multiple uncertainties is a challenge for clinicians [21Ghosh AK. Understanding medical uncertainty: a primer for physicians J Assoc Physicians India 2004; 52(9): 739-42., 22Moskowitz AJ, Kuipers BJ, Kassirer JP. Dealing with uncertainty, risks, and tradeoffs in clinical decisions. A cognitive science approach Ann Intern Med 1988; 108(3): 435-9.], so arguably even more so for planners, designers or implementers of HI solutions [23Wears RL, Berg M. Computer technology and clinical work: still waiting for Godot JAMA 2005; 293(10): 1261-3., 24Wears RL, Berg M. Computers and clinical work-reply JAMA 2005; 294(2): 182-b.]. In particular, the NHS is a paradox: nominally a centrally-controlled monolith but in fact a coalition of competing confederacies of autonomous interests.

We argue that this irreducible complexity means that a socio-technical approach is vital to the success of innovations and transformations that are based on HI. The socio-technical approach would help to negotiate these complexities using principles such as minimal process specification, multi-stakeholder participation, user leadership, local adaptability and holistic evaluation [14Clegg C. Sociotechnical principles for system design Appl Ergon 2000; 31(5): 463-77.].

Unfortunately, knowledge of socio-technical principles and their vital importance in HI does not seem to be widespread among practitioners. The socio-technical assessment tool for health informatics (STAT-HI) proposed here aims to help bridge the gap between research and practice by distilling key factors of good practice into a simple, readily accessible and applicable form.

The tool is expressed as a reflective checklist to validate the socio-technical soundness of an HI implementation approach. It does not aim to check every aspect of project fitness for purpose.

This approach seeks to utilize the ‘checklist effect’. Merely having a structured set of tasks or questions can in itself improve outcomes. This effect is so strong that it is a well-known source of bias in studies that compare interventions that use any checklist-based approach (whatever its merits) against those that do not [25Friedman C, Wyatt J. Evaluation methods in medical informatics. London: Springer-Verlag 1997.]. Checklists have proved very powerful in improving aspects of clinical practice [26Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU N Engl J Med 2006; 355(26): 2725-32.].

The checklist was conceived as a categorized summary of principles derived from a synthesis of critical success factors identified in the literature. It emulates the approach and certain principles of the AHRQ [27Cusack C, Byrne C, Hook J, McGowan J, Poon E, Zafar A. Health Information Technology Evaluation Toolkit: 2009 Update Rockville, MD: Agency for Healthcare Research and Quality 2009 June; AHRQ Publication No. 09-0083-EF], GEP-HI [28Nykänen P, Brender J, Ammenwerth E, et al. Guidelines for Good Evaluation Practices in Health Informatics (v06). [electronic document] Available from: http://iig.umit.at/efmi/gep-hi/GEP_HI_v016_Dec012008.doc 2008 [cited 2009 4 September];] and STARE-HI [29Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykanen P, Rigby M. STARE-HI--Statement on reporting of evaluation studies in Health Informatics Int J Med Inform 2009; 78(1): 1-9.] standards for designing and reporting system evaluation in health informatics.

The primary sources used were seminal UK reports on reasons for success or failure in major IT projects [30Royal Academy of Engineering, British Computer Society. The challenges of complex IT projects [electronic document] Available from: http://www.bcs.org/upload/pdf/complexity.pdf 2004 [cited 2009 November 9];, 31UK National Audit Office. Delivering successful IT-enabled business change [electronic document] Available from: http://www.nao.org.uk/publications/0607/delivering_successful_it-enabl.aspx 2006 [cited 2009 November 9];] and a selection of HI studies [32Ozdas A, Miller RA. Care provider order entry (CPOE): a perspective on factors leading to success or to failure Yearb Med Inform 2007; 128-37.-39Physician Order Entry Team. POET Recommendations [electronic document] Available from: http://www.cpoe.org/ 2007 [cited 2009 November 11];] encountered in an ongoing literature review of health informatics theory by the present authors and colleagues [40Scott PJ, Briggs JS, Wyatt JC, Georgiou A. How Important is Theory in Health Informatics?. A Survey of the UK Health Informatics Academic Community Manuscript submitted for publication 2010.]. This selection does not purport to be comprehensive or to form a systematic review of socio-technical principles in HI, but is presented as a starting point for discussion and development. The sources were selected on face validity as offering immediately relevant and directly applicable socio-technical assessment criteria.

The checklist emphasises particular elements of programme governance, cultural factors, the approach to changes in workflow and practice and the need for strong and balanced clinical leadership and governance. The headings echo the critical success factors identified by Whetton [17Whetton S. Health Informatics: A socio-technical perspective. Oxford: OUP 2005.] (see Box 2) and Georgiou & Westbrook’s ten questions to consider before implementing computerized physician order entry (CPOE) [41Georgiou A, Westbrook J. Computerised Order Entry Systems and Pathology Services - A Synthesis of the Evidence Clin Biochem Rev 2006; 27(2): 79-87.] (see Box 3).

The STAT-HI checklist is presented as a tool for project/programme managers, project/programme boards and other professionals who provide external assurance and audit functions.

It is suggested that a required reading list should be promulgated to HI practitioners alongside this checklist, comprising eight books, papers and electronic sources [30Royal Academy of Engineering, British Computer Society. The challenges of complex IT projects [electronic document] Available from: http://www.bcs.org/upload/pdf/complexity.pdf 2004 [cited 2009 November 9];-32Ozdas A, Miller RA. Care provider order entry (CPOE): a perspective on factors leading to success or to failure Yearb Med Inform 2007; 128-37., 34Berg M. Medical work and the computer-based patient record: a sociological perspective Methods Inform Med 1998; 37(3): 294-301., 37Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry Int J Med Inform 2007; 76(Suppl 1): 21-7.-39Physician Order Entry Team. POET Recommendations [electronic document] Available from: http://www.cpoe.org/ 2007 [cited 2009 November 11];, 45UK Office of Government Commerce. Managing Successful Programmes London: OGC 2003.].

The reading list supports the aim of the checklist to be reflective, rather than simply a tick-box exercise. The checklist is an opportunity to learn, to enlarge one’s worldview and to change practice. Schön [46Schön D. The reflective practitioner: how professionals think in action Aldershot: Ashgate 1991.] argued that reflective practice goes beyond problem-solving by considering problem setting: has it been ‘named and framed’ rightly, or is evidence emerging that it needs to be rethought?

Therefore we suggest that a reviewer (whatever their professional role) making an assessment using STAT-HI should give narrative justification for their considered responses to each of the questions, adducing data to substantiate their commentary as necessary to satisfy both their own ethical standard and the independent critical judgement of the relevant project/programme assurance bodies.

By definition, this checklist utilizes existing evidence and prior good practice recommendations. Its only claim to originality is the condensation and annotated checklist presentation of key guidance and the suggestion of a specific minimum knowledge base for HI practitioners.

Arguably there are socio-technical principles embedded in numerous extant methods, even if not explicitly. Examples of this include the OGC programme management guidelines, Soft Systems Methodology and the ETHICS method.

OGC’s Managing Successful Programmes [45UK Office of Government Commerce. Managing Successful Programmes London: OGC 2003.] acknowledges that change programmes involve ambiguity and the need to establish both organizational readiness for change and capability to manage and deliver change. Programmes are described as led by vision not specification. They are most effective when issues are debated freely and risks evaluated openly. The programme team needs staff with the relevant skills and experience to manage cultural and people issues involved in the business change.

Checkland’s Soft Systems Methodology is a process of enquiry that accepts and examines the multiple perceived realities of different stakeholders in a problem scenario. It envisages a group learning cycle that seeks to discover feasible improvements that can be accommodated within the continually re-negotiated social interpretations of individuals and organizations. Its philosophy directly rejects the idea that its conceptual models represent descriptions of an absolute reality, but regards “systems” as devices to think about the purposeful activities of various actors and thus formulate questions of the problem situation [47Checkland P, Poulter J. Learning for action: a short definitive account of soft systems methodology and its use for practitioners, teachers and students Chichester: John Wiley 2006.]. So if used for HI, the soft systems approach might in a sense be regarded as an application of socio-technical principles but with a more strongly constructivist interpretation of what we mean by “system”.

Mumford devised the ETHICS method (Effective Technical and Human Implementation of Computer-based Systems) as an explicitly socio-technical approach [15Mumford E. Systems design: ethical tools for ethical change. Basingstoke: Macmillan 1996.-16Mumford E. Redesigning human systems. Hershey, PA:: IRM Press 2003.]. It emphasizes participatory diagnosis of problems, setting objectives for both efficiency and work satisfaction and ensuring compatibility of the technical and organizational systems. Mumford’s extensive socio-technical work seems to be little known outside academia, particularly since the 1980s [11Mumford E. The story of socio-technical design: reflections on its successes, failures and potential Inform Syst J 2006; 16(4): 317-42.].

Given the above, we echo Cobb’s paradox: "We know why projects fail, we know how to prevent their failure – so why do they still fail?" [30Royal Academy of Engineering, British Computer Society. The challenges of complex IT projects [electronic document] Available from: http://www.bcs.org/upload/pdf/complexity.pdf 2004 [cited 2009 November 9];].

One factor is that health informatics and project management are not controlled professions with mandatory regulatory standards. Hence, the professional competence of some NHS HI project managers is limited to general IT project experience, which tends to bring the implicit but incorrect view that HI is "just another" industry sector. This has been another general criticism of the NPfIT “commodity” approach to HI. If anything, the unregulated nature of project management and health informatics means that their need for guidance is even greater than a formally controlled profession.

A converse aspect of the problem is information overload. There is simply so much guidance available on project and programme management in healthcare. In some respects this parallels medical practice, where for many years it has been recognized that practicing clinicians cannot be expected to keep track of every new piece of research evidence. Instead, important new knowledge is highlighted through summary publications, clinical guidelines and professional standards. In fact, the latency inherent in the medical profession is such that even solid research evidence only changes clinical practice slowly, patchily and when multiple interventions are used for reinforcement [38Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality J Am Med Inform Assoc 2003; 10(6): 523-30., 48Wyatt J. Clinical knowledge and practice in the information age: a handbook for health professionals. London: RSM Press 2001.].

By contrast, HI benefits and project management methodologies do not have a robust evidence base but are largely aspirational [42Clamp S, Keen J. Electronic health records: is the evidence base any use? In: Bryant J, Ed. Proceedings of Healthcare Computing Harrogate: BCS Health Informatics Forum. 2006.] or normative [49UK Office of Government Commerce In: Managing Successful Projects with PRINCE2. London: OGC 2005.]. In fact there are legitimate grounds for caution with respect to HI, given the evidence that products are not equal [50Murff HJ, Kannry J. Physician satisfaction with two order entry systems J Am Med Inform Assoc 2001; 8(5): 499-509.], things can be made worse [51Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF. High rates of adverse drug events in a highly computerized hospital Arch Intern Med 2005; 165(10): 1111-6.] and that the method of implementation can make all the difference to success or failure [34Berg M. Medical work and the computer-based patient record: a sociological perspective Methods Inform Med 1998; 37(3): 294-301., 52Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study BMJ 2005; 331(7528): 1313-6.].

At times HI is used instrumentally, beyond its functional purposes, to enforce “improvements” in ways of working as defined by the project sponsors [53Broadhurst K, Wastell D, White S, et al. Performing 'Initial Assessment': Identifying the Latent Conditions for Error at the Front-Door of Local Authority Children's Services Br J Soc Work [electronic document] Available from: http://bjsw.oxford journals.org/cgi/content/abstract/bcn162v1 2009 [cited 2009 11 December];]. This adds yet another layer of complexity and potential opposition, especially if the merits of the given changes are disputed.

Of course, the invidious effects of politics and commercial interests cannot be ignored in this context. The NHS is unavoidably part of the political game, given its funding and structure. A taxpayer-funded IT programme has legitimate expectations of optimum progress from both its political masters and its commercial suppliers. But when these interests seem to sacrifice good practice for the sake of apparent progress or simply meeting contractual milestones, then the taxpayers have sound reason for concern.

It must also be stressed that the anticipated benefits of planned change in healthcare systems, particularly when based on HI, are not deterministic. The inherent complexity of the socio-technical system in healthcare means that apparently minor changes can have profound unpredictable consequences [18Plsek P. Redesigning healthcare with insights from the science of complex adaptive systems In: Committee on Quality of Health Care in America, Ed Crossing the quality chasm. Washington DC: National Academy Press 2001., 37Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry Int J Med Inform 2007; 76(Suppl 1): 21-7.]. Indeed, it has been argued that any ICT “solution” actually adds complexity, and hence risk, by increasing the number of information and process interrelationships in the world [54Hanseth O. Integration-complexity-risk: the making of information systems out-of-control In: Hanseth O, Ciborra C, Eds. Risk complexity and ICT. Cheltenham: Edward Elgar 2007.].

NHS Connecting for Health (CFH), the English Department of Health agency responsible for NPfIT, publishes an “Organisational Readiness Assurance Guide” for HI planning [55NHS Connecting for Health. CFH Organisational Readiness Assurance Guide [electronic document] Available from: http://nww.connectingforhealth.nhs.uk/imtplanning/aguide.pdf 2008. [Only accessible within NHS network]]. This document seems to assume a linear model of change that comprises work process redesign, system deployment and establishing a new status quo. It does not address unintended consequences or complex adaptive change. Its subtitle “Are you ready?” reinforces the concept of the organization or individual as passively receptive to a “deployment”. It does suggest that new working practices and roles be subjected to an impact analysis and requires that both line managers and “stakeholders” agree to the new working practices and roles, but does not specify whether that includes the clinicians themselves. Arguably, its helpful direction on applying lessons learned from other HI implementations itself implies that the socio-technical perspective needs greater emphasis.

The genesis and focus of this paper has been on HI in the UK. However, the essential principles discussed here would intuitively seem rather general across the global HI domain.

At best, this proposed tool can offer a selection of key messages and maxims to practitioners. This is significantly less than a formal theory of HI programme management, but is arguably an improvement on the status quo.

Obviously, this proposed tool will not remove political or contractual pressure to impose systems to meet arbitrary deadlines. However, it could provide a mechanism to create or channel balancing feedback pressure within or upon the governance structures of HI programmes where participants have concerns that good practice may be compromised by inappropriate commercial or governmental timescales.

The suggested approach will be debated and the content and format tested with the UK Faculty for Health Informatics. If the tool seems to have face validity based on this expert consensus, the aim will be to apply it to current or recent HI projects to measure its utility. One extension may be to devise a numerically scored assessment rating. Further thought is needed about how in practice the rating would be conducted for it to be credible, and how any proposed remedial action would be validated. Further essential content for the checklist may be added from analysis of the recently published AMIA collection of case studies of failed HI projects [56Leviss J, Gugerty B, Kaplan B, Keenan G, Ozeran L, Rose E, Eds. H.T. or miss: lessons learned from health information technology implementations Chicago IL: AHIMA Press 2010.].