The Open Medical Informatics Journal




(Discontinued)

ISSN: 1874-4311 ― Volume 13, 2019

A Sociotechnical Approach to Evaluating the Impact of ICT on Clinical Care Environments



Julie Li
Health Informatics Research and Evaluation Unit, Faculty of Health Sciences, The University of Sydney, 75 East St Lidcombe, NSW 1825, Australia

Abstract

Introduction: Process-supporting information technology holds the potential to increase efficiency, reduce errors, and alter professional roles and responsibilities in a manner which allows improvement in the delivery of patient care. However, clashes between the model of health care work inscribed in these tools with the actual nature of work has resulted in staff resistance and decreased organisational uptake of ICT, as well as the facilitation of unexpected and negative effects in efficiency and patient safety. Sociotechnical theory provides a paradigm against which workflow and transfusion of ICT in healthcare could be better explored and understood.

Design: This paper will conceptualise a formative, multi-method longitudinal evaluation process to explore the impact of ICT with an appreciation of the relationship between the social and technical systems within a clinical department.

Method: Departmental culture, including clinical work processes and communication patterns will be thoroughly explored before system implementation using both quantitative and qualitative research methods. Findings will be compared with post implementation data, which will incorporate measurement of safety and workflow efficiency indicators.

Discussion: Sociotechnical theory provides a paradigm against which workflow and transfusion of ICT in healthcare could be better explored and understood. However, sociotechnical and multimethod approaches to evaluation do not exist without criticism. Inherent in the protocol are limitations of sociotechnical theory and criticism of the multimethod approach; testing of the methodology in real clinical settings will serve to verify efficacy and refine the process.

Keywords: Sociotechnical theory, mixed method evaluation, medical informatics, healthcare evaluation mechanisms.


Article Information


Identifiers and Pagination:

Year: 2010
Volume: 4
First Page: 202
Last Page: 205
Publisher Id: TOMINFOJ-4-202
DOI: 10.2174/1874431101004010202

Article History:

Received Date: 30/11/2009
Revision Received Date: 21/1/2010
Acceptance Date: 25/1/2010
Electronic publication date: 21 /10/2010
Collection year: 2010

Article Metrics:

CrossRef Citations:
0

Total Statistics:

Full-Text HTML Views: 651
Abstract HTML Views: 844
PDF Downloads: 273
Total Views/Downloads: 1768

Unique Statistics:

Full-Text HTML Views: 460
Abstract HTML Views: 576
PDF Downloads: 141
Total Views/Downloads: 1177
Geographical View

© Julie Li; Licensee Bentham Open.

open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.


* Address correspondence to this author at the Health Informatics Research and Evaluation Unit, Faculty of Health Sciences, The University of Sydney, 75 East St Lidcombe, NSW 1825, Australia; Tel: +61 2 9351 9989; Fax: +61 2 9351 9676; E-mail: julie.li@sydney.edu.au




INTRODUCTION

Process-supporting information technology holds the potential to increase efficiency, reduce errors, and alter professional roles and responsibilities in a manner which allows improvement in the delivery of patient care [1Fonkych K, Taylor R. The state and pattern of health information technology adoption, Rand Corporation, Santa Monica 2005.]. However, despite large-scale investments in Information Communication Technology (ICT), uptake has been slow [1Fonkych K, Taylor R. The state and pattern of health information technology adoption, Rand Corporation, Santa Monica 2005., 2Ash JS, Gorman PN, Seshadri V, et al. Computerized physician order entry in U.S. hospitals: results of a 2002 survey J Am Med Inform Assoc 2004; 11(2 ): 95-.] whilst recent evidence of ICT-induced errors has called for a more critical examination of the impact of such technology in healthcare [3Koppel R, Metlay J, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors J Am Med Assoc 2005; 293(10 ): 1197-203.-6Perrow C. Normal accidents: Living with high-risk technologies 1999.].

Grounded in both social theory and Information Technology (IT), sociotechnical approaches to ICT evaluation focuses on the interrelation between technology and its social environment [7Berg M, Aarts J, Van Der Lei J. ICT in health care: sociotechnical approaches Methods Inform Med 2003; 42(4 ): 297-301.]. Over time, ICT implementation models have developed an emphasis on the alignment between the clinical context where clinical work is carried out and the technology designed to improve the delivery of care [8Goodhue D, Thompson R. Task-technology fit and individual performance MIS Q 1995; 00: 213-36.-10Ammenwerth E, Iller C, Mahler C, Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study BMC Med Inf Decis Mak 2006; 6: 3-0.], as well as the contextual differences both between organisations and within – at departmental and individual levels [11Callen J, Braithwaite J, Westbrook J. Contextual implementation model: a framework for assisting clinical information system implementations J Am Med Inform Assoc 2008; 15(2 ): 255-62.]. This paper will propose a methodology for the sociotechnical evaluation of the impact of a computerised information system on a clinical care department.

WHY SOCIOTECHNICAL THEORY?

Contemporary health organisations are complex, composed of deeply interdependent and interrelated social and technical elements where changes to one aspect will affect the other [12Wears RL, Berg M. Computer technology and clinical work: still waiting for godot JAMA 2005; 293(10 ): 1261-3.]; the introduction of any new technology into a setting will entail innovation in clinical roles, work processes, and culture change [13Coiera E. Four rules for the reinvention of health care BMJ 2004; 328(7449 ): 1197-9.-15Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: Findings from a mixed methods exploration Int J Med Inform 2009; 78(Suppl1): S69-76.] whilst the attitudes and utility of technology is simultaneously socially shaped [13Coiera E. Four rules for the reinvention of health care BMJ 2004; 328(7449 ): 1197-9.]. Rather than viewing computer systems as isolated entities within their intended clinical environment, the contemporary perspective portrays systems as an active component of the clinical team, constantly cooperating and dynamically interacting with the clinical staff and organisational routines [12Wears RL, Berg M. Computer technology and clinical work: still waiting for godot JAMA 2005; 293(10 ): 1261-3., 16Anderson J, Aydin C. Evaluating the organizational impact of healthcare information systems 2005.].

The sociotechnical approach is one which seeks to identify the dynamics between technology and the social, professional, and cultural environment in which it is used [17Whetton S, Ed. Health informatics: a socio-technical perspective Melbourne: Oxford University Press 2005.]. The limitations of a purely technocentric approach to systems evaluation became apparent when early implementation projects were less than successful and purported benefits of ICT rarely realised [17Whetton S, Ed. Health informatics: a socio-technical perspective Melbourne: Oxford University Press 2005.-19Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised health information systems: hard lessons still to be learnt BMJ 2003; 326(7394 ): 860-3.]. Causes of implementation failure extend beyond poor system design to the erroneous perceptions and theories about how medical work is conducted [12Wears RL, Berg M. Computer technology and clinical work: still waiting for godot JAMA 2005; 293(10 ): 1261-3., 20Aarts J, Gorman P. IT in health care: sociotechnical approaches "To Err is System" Int J Med Inform 2007; 76(Suppl 1): S1-3., 21Aarts J, Ash J, Berg M. Extending the understanding of computerized physician order entry: Implications for professional collaboration, workflow and quality of care Int J Med Inform 2007; 76: 4-13.]. Clashes between the model of health care work inscribed in these tools with the actual nature of work has resulted in staff resistance and decreased organisational uptake of ICT, as well as the facilitation of unexpected and negative effects in efficiency and patient safety [3Koppel R, Metlay J, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors J Am Med Assoc 2005; 293(10 ): 1197-203., 4Han Y, Carcillo J, Venkataraman S, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system Pediatrics 2005; 116(6 ): 1506-2., 15Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: Findings from a mixed methods exploration Int J Med Inform 2009; 78(Suppl1): S69-76., 18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004.]. Sociotechnical theory provides a paradigm against which workflow and perfusion of ICT in healthcare could be better explored and understood.

DESIGN

Any form of ICT evaluation in health may be distinguished as either summative or formative. Summative evaluations determine the overall quality or value of an intervention and have traditionally followed the objectivist view which dictates that the merits and worth of ICT should and could be quantified [18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004., 22Scriven M. The logic and methodology of evaluation, Edgepress, Inverness.CA 1981.]. In healthcare, such evaluations of ICT are conducted post hoc to account for the impacts of the system such as financial benefits or system effectiveness in terms of clinical outcomes [18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004.]. Dominating the literature on health ICT evaluation, ICT is often conceptualised as the “intervention” in summative studies which allude, as in medicine, to the Randomised Controlled Trial (RCT) model of evaluation as the “gold standard” of evaluation [18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004., 23Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism Int J Med Inform 2001; 64(1 ): 39-56.].

An issue with ICT evaluation in health is the complexity of the evaluation object [24Ammenwerth E, Gräber S, Herrmann G, Bürkle T, König J. Evaluation of health information systems—problems and challenges Int J Med Inform 2003; 71(2-3 ): 125-35.]. The deep integration of ICT with the culture and work practices of the healthcare organisation requires an evaluative focus which expands beyond only software and hardware to the surrounding clinical situation and associated processes to capture the diverse and diffuse nature of system effects in the healthcare setting [24Ammenwerth E, Gräber S, Herrmann G, Bürkle T, König J. Evaluation of health information systems—problems and challenges Int J Med Inform 2003; 71(2-3 ): 125-35.]. Whilst RCT-type studies are valuable for determining the cause and impact of a pre-specified effect, it assumes possibility of isolating system functioning from the social processes that surround it, whilst reality suggests that the design, implementation, and operation of ICT are deeply influenced by organisational and social factors [23Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism Int J Med Inform 2001; 64(1 ): 39-56.].

Formative evaluations are conducted during the development of a product for the purpose of system improvement [22Scriven M. The logic and methodology of evaluation, Edgepress, Inverness.CA 1981.]. The increase in qualitative studies of health ICT has exhibited a shift towards the subjectivist theoretical orientation to systems evaluation which asserts the responsibility of social and organisational aspects in shaping the outcomes of objective measurement and endorsing a “multimethod” approach for both summative and formative evaluations [18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004.]. The mixed or multimethod approach to health ICT evaluation is one which views qualitative and quantitative research methods as complementary [23Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism Int J Med Inform 2001; 64(1 ): 39-56., 25Westbrook JI, Braithwaite J, Georgiou A. Multimethod evaluation of information and communication technologies in health in the context of wicked problems and sociotechnical theory J Am Med Inform Assoc 2007; 14(6 ): 746-55., 26Stoop AP, Berg M. Integrating quantitative and qualitative methods in patient care information system evaluation: guidance for the organizational decision maker Methods Inform Med 2003; 42(4 ): 458-62.]; the combination of methods allows exploration of the what, why, and how of a social phenomenon that qualitative methods can address, and the size, extent or duration (how much) of certain phenomena that quantitative methods establish [26Stoop AP, Berg M. Integrating quantitative and qualitative methods in patient care information system evaluation: guidance for the organizational decision maker Methods Inform Med 2003; 42(4 ): 458-62.]. Pleiotropic healthcare interventions such as ICT create a hybrid sociotechnical situation requiring a range of integrated multi-perspective research methods to achieve a holistic understanding [23Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism Int J Med Inform 2001; 64(1 ): 39-56., 27Scott P, Briggs J. A pragmatist argument for mixed methodology in medical informatics J Mixed Methods Res 2009; 3(3 ): 223-0., 28Kaplan B, Shaw N. Future directions in evaluation research: people, organizational, and social issues Methods Inform Med 2004; 43(3 ): 215-31.].

ICT is also a dynamic and emerging process, continually evolving across the multiple phases of the implementation process [11Callen J, Braithwaite J, Westbrook J. Contextual implementation model: a framework for assisting clinical information system implementations J Am Med Inform Assoc 2008; 15(2 ): 255-62., 23Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism Int J Med Inform 2001; 64(1 ): 39-56.]. A longitudinal study design which follows the target of study over a substantial period of time will allow appreciation of this aspect by providing a means of monitoring changes caused by the system within the context it operates [22Scriven M. The logic and methodology of evaluation, Edgepress, Inverness.CA 1981., 24Ammenwerth E, Gräber S, Herrmann G, Bürkle T, König J. Evaluation of health information systems—problems and challenges Int J Med Inform 2003; 71(2-3 ): 125-35.].

This paper will conceptualise a formative, multi-method longitudinal evaluation process to explore the impact of ICT with an appreciation of the relationship between the social and technical systems within a clinical department.

METHOD

Pre System Implementation

Objective

1. To explore the context of the clinical department.

Process

Implicit in the sociotechnical approach to systems evaluation is consideration for the context in which the system is implemented [29Pasmore W. Social science transformed: the socio-technical perspective Hum Relat 1995; 48(1 ): 1-21.]. Differing socio-organisational settings influence ICT adoption processes and system impact [10Ammenwerth E, Iller C, Mahler C, Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study BMC Med Inf Decis Mak 2006; 6: 3-0., 11Callen J, Braithwaite J, Westbrook J. Contextual implementation model: a framework for assisting clinical information system implementations J Am Med Inform Assoc 2008; 15(2 ): 255-62., 30Callen J, Braithwaite J, Westbrook J. Cultures in hospitals and their influence on attitudes to, and satisfaction with, the use of clinical information systems Soc Sci Med 2007; 65(3 ): 635-9.-33Peute L, Aarts J, Bakker P, Jaspers M. Anatomy of a failure: A sociotechnical evaluation of a laboratory physician order entry system implementation Int J Med Inform 2010; 74(4 ): 58-70.]. Departmental culture, including roles, clinical work processes and communication patterns will be thoroughly explored before system implementation to identify focus areas and facilitate interpretation of findings post implementation. Focus groups will be held pre system implementation with a purposive sample of participants across all clinical staff groups to obtain a broad representative cross-section. Discussion will concentrate on how work is currently conducted, issues of concern and expectations of the electronic system to gain a general understanding of the nature and needs of the department. Individual roles as well as the perceived roles and contributions of members of other staff groups will be explored and triangulated to establish a sense of departmental culture and dynamics. Significant points or themes arising from analysis of transcripts will be further explored during in-depth interviews with relevant participants or key informants identified from initial focus groups.

Whilst interview and focus groups examine how a situation is perceived, they are limited in their ability to understand and explain behaviour [34Norman D. Emotional Design: Why we love (or hate) everyday things 2004.]. Observations of actual work processes in real situations divulge more about individual work, successes, failures, and preferences than consultation with participants alone [34Norman D. Emotional Design: Why we love (or hate) everyday things 2004.]. Emerging themes from interviews and focus groups will also be explored through sufficient hours of non participant observation. This will be supplemented with video recordings which allow reviewing of specific work processes/behaviour multiple times, careful transcription and re-transcription of events to reveal and analyse “fine-grained, ‘seen but unnoticed’” aspects of conduct otherwise unidentified from the grosser examples of work phenomena obtained from plain observation alone [35Greatbatch D, Murphy E, Dingwall R. Evaluating medical information systems: ethnomethodological and interactionist approaches Health Serv Manage Res 2001; 14(3 ): 181-91.]. Non-participant observations and video recordings will follow certain individuals or teams during routine work processes with focus on specific items raised during interviews and focus groups.

Whilst qualitative research methods such as interviews, focus groups, and observations yield context-dependent data from the perspective of the participants within their organisational or departmental context, quantitative means for measuring social and departmental characteristics also exist, and may be used in conjunction with qualitative measures to strengthen robustness of data through triangulation [18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004.]. Social network analysis examines communication patterns and interactions between professional groups to elicit the patterns of connections that occur between staff [36Heng HKS, McGeorge WD, Loosemore M. Beyond strategy: Exploring the brokerage role of facilities manager in hospitals J Health Org Manag 2005; 19: 16-31.]. This may be further quantified using the Team Climate Inventory (TCI) which measures team cohesiveness and support for work innovation which serves to predict the likelihood of departmental acceptance and successful adoption of ICT [37Ouwens M, Hulscher M, Akkermans R, Hermens R, Grol R, Wollersheim H. The team climate inventory: application in hospital teams and methodological considerations Qual Saf Health Care 2008; 17(4 ): 275-0.]. For the purposes of this study the entire department should be conceptualised as the team.

Post System Implementation

Objectives

  1. To determine the impact of the clinical information system on staff functioning and departmental dynamics
  2. To quantify the impact of the clinical information system on patient safety, departmental productivity and efficiency indicators

Process

Evaluation will maintain focus on the impact of ICT on departmental culture, work practices and communication post implementation. Focus groups, interviews, and observations conducted pre system implementation will be repeated post implementation to identify changes to the previously defined culture, roles, communication patterns and work processes. Questions will be informed by existing literature on the impact of information systems on culture and work in conjunction with pre implementation findings, including specific areas of concern identified from pre implementation study unique to the department. Social network surveys will be redistributed at this stage to identify changes in communication patterns and individual roles in the exchange of information in the network.

A central argument for the implementation of computerised patient information systems is their potential for improving patient safety and work practice efficiency [1Fonkych K, Taylor R. The state and pattern of health information technology adoption, Rand Corporation, Santa Monica 2005.]. Key performance indicators for safety and productivity (e.g. medication errors [25Westbrook JI, Braithwaite J, Georgiou A. Multimethod evaluation of information and communication technologies in health in the context of wicked problems and sociotechnical theory J Am Med Inform Assoc 2007; 14(6 ): 746-55.], diagnostic turnaround times [38Georgiou A, Westbrook J, Braithwaite J, et al. When requests become orders—A formative investigation into the impact of a computerized physician order entry system on a pathology laboratory service Int J Med Inform 2007; 76(8 ): 583-91.], and patient waiting times [39Boyce NW, Boyce NW. Potential pitfalls of healthcare performance indicators Med J Aust 2002; 177(5 ): 229-30.]) exist from evaluation literature in the field, and will be selected and measured in conjunction with context-dependent issues or indicators identified from analysis of qualitative data both pre and post system implementation. To elucidate impacts of the computerised system, data from both before and after implementation of the system will be elicited from relevant administration databases as well as paper documentation. Quantitative measurements of indicator data supplement qualitative findings, adding rigour and validation whilst qualitative data place quantitative findings in an appropriate social setting that allows interpretation and a broader understanding of their implications [40Pawson R. A measure for measures: a manifesto for empirical sociology 1989.].

ICT implementation is multi-staged [11Callen J, Braithwaite J, Westbrook J. Contextual implementation model: a framework for assisting clinical information system implementations J Am Med Inform Assoc 2008; 15(2 ): 255-62., 23Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism Int J Med Inform 2001; 64(1 ): 39-56.]. Data collection will be carried out immediately post implementation and continue at appropriate intervals across the various stages throughout the implementation process to monitor impacts and inform system customisation and development to improve system integration into existing work processes and culture. Findings and recommendations should be considered and formulated in light of influence from complex external factors such as resource limitations, government policy, influence from different stakeholders, the economic environment, and features of the health system and the information technology industry [11Callen J, Braithwaite J, Westbrook J. Contextual implementation model: a framework for assisting clinical information system implementations J Am Med Inform Assoc 2008; 15(2 ): 255-62., 18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004.].

Study Outcomes

  1. Quantification of the effects of a clinical information system on a range of safety, productivity, and efficiency indicators
  2. Identification of the underlying mechanisms which determine optimal systems functioning within the clinical department

DISCUSSION

The sociotechnical approach to ICT evaluation appreciates the highly complex healthcare setting and emphasises the relationship between the social and technical impacts of clinical information systems. ICT represent technical artefacts which embody the implicit theories about how work is conducted; success of the artefact in real clinical situations relies upon the accuracy and applicability of these theories [41Woods D. Designs are hypotheses about how artifacts shape cognition and collaboration Ergonomics 1998; 41(2 ): 168-73.]. ICT evaluation tools [42Nykänen P, Brender J, Ammenwerth E. Guidelines for Good Evaluation Practices in Health Informatics (v0.16) 2008 [cited 2009 Jan 5]: Available from: http://iig.umit.at/efmi/gep-hi/GEP_HI_v016_Dec012008.doc. , 43Cusack C, Byrne C, Hook J, McGowan J, Poon E, Zafar A. Health information technology evaluation toolkit: 2009 Update. Rockville, MD: Agency for Healthcare Research and Quality Report No.: AHRQ Publication No. 09-0083-EF. June 2009 2009.] exist to guide the process of planning and execution of evaluation projects, with which the proposed protocol may be further developed and fine-tuned to accommodate individual settings; the methodology outlined in the paper supplements existing work in its offering of a unique, practical evaluation procedure which combines the theory and practice of the sociotechnical perspective to attain a more holistic understanding of the impacts of ICT and to ultimately achieve successful implementation.

However, sociotechnical theory does not exist without criticism. Examination of both the social and technical systems within a clinical department seeks to determine the different contextual factors operating in separate settings in order to identify the latent mechanisms that can influence outcomes [18Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004.]. Berg [44Berg M. Patient care information systems and health care work: a sociotechnical approach Int J Med Inform 1999; 55(2 ): 87-101.] cautions that understanding the interdependency of the social and technical systems within a particular context is insufficient for reconciling the social and technical aspects within that environment. Similarly, multi-method approaches to evaluation could potentially produce contradictory findings which do not, in fact, facilitate mutual interpretation of results produced from the separate methods [28Kaplan B, Shaw N. Future directions in evaluation research: people, organizational, and social issues Methods Inform Med 2004; 43(3 ): 215-31.]. The proposed methodology risks inheriting such limitations; efficacy of the process remains to be verified through independent field testing.

CONCLUSION

Sociotechnical theory provides a paradigm against which workflow and transfusion of ICT in healthcare could be better explored and understood. Incorporating both interactionist and ethnomethodological methods, this paper has outlined a formative, multi-method, longitudinal model for ICT evaluation which aims to inform interpretation of objective system effects in light of how the IT is conceptualised and viewed by users, as well as its impact on staff work practices within the unique context of the clinical department. However, inherent in the protocol are limitations of sociotechnical theory and criticism of the multimethod approach; testing of the methodology in real clinical settings will serve to verify efficacy and refine the process.

REFERENCES

[1] Fonkych K, Taylor R. The state and pattern of health information technology adoption, Rand Corporation, Santa Monica 2005.
[2] Ash JS, Gorman PN, Seshadri V, et al. Computerized physician order entry in U.S. hospitals: results of a 2002 survey J Am Med Inform Assoc 2004; 11(2 ): 95-.
[3] Koppel R, Metlay J, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors J Am Med Assoc 2005; 293(10 ): 1197-203.
[4] Han Y, Carcillo J, Venkataraman S, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system Pediatrics 2005; 116(6 ): 1506-2.
[5] Garg AX, Adhikari NKJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review JAMA 2005; 293(10 ): 1223-38.
[6] Perrow C. Normal accidents: Living with high-risk technologies 1999.
[7] Berg M, Aarts J, Van Der Lei J. ICT in health care: sociotechnical approaches Methods Inform Med 2003; 42(4 ): 297-301.
[8] Goodhue D, Thompson R. Task-technology fit and individual performance MIS Q 1995; 00: 213-36.
[9] Aarts J, Peel V, Wright G. Organizational issues in health informatics: a model approach Int J Med Inform 1998; 52(1-3 ): 235-42.
[10] Ammenwerth E, Iller C, Mahler C, Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study BMC Med Inf Decis Mak 2006; 6: 3-0.
[11] Callen J, Braithwaite J, Westbrook J. Contextual implementation model: a framework for assisting clinical information system implementations J Am Med Inform Assoc 2008; 15(2 ): 255-62.
[12] Wears RL, Berg M. Computer technology and clinical work: still waiting for godot JAMA 2005; 293(10 ): 1261-3.
[13] Coiera E. Four rules for the reinvention of health care BMJ 2004; 328(7449 ): 1197-9.
[14] Ash JS, Sittig DF, Campbell E, et al. An unintended consequence of CPOE implementation: shifts in power, control, and autonomy: 2006: Proceedings of the AMIA Annual Symposium. Washington DC, United States of America 2006.
[15] Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: Findings from a mixed methods exploration Int J Med Inform 2009; 78(Suppl1): S69-76.
[16] Anderson J, Aydin C. Evaluating the organizational impact of healthcare information systems 2005.
[17] Whetton S, Ed. Health informatics: a socio-technical perspective Melbourne: Oxford University Press 2005.
[18] Berg M. Health information management: integrating information technology in health care work. Routledge, London, 2004.
[19] Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised health information systems: hard lessons still to be learnt BMJ 2003; 326(7394 ): 860-3.
[20] Aarts J, Gorman P. IT in health care: sociotechnical approaches "To Err is System" Int J Med Inform 2007; 76(Suppl 1): S1-3.
[21] Aarts J, Ash J, Berg M. Extending the understanding of computerized physician order entry: Implications for professional collaboration, workflow and quality of care Int J Med Inform 2007; 76: 4-13.
[22] Scriven M. The logic and methodology of evaluation, Edgepress, Inverness.CA 1981.
[23] Kaplan B. Evaluating informatics applications—some alternative approaches: theory, social interactionism, and call for methodological pluralism Int J Med Inform 2001; 64(1 ): 39-56.
[24] Ammenwerth E, Gräber S, Herrmann G, Bürkle T, König J. Evaluation of health information systems—problems and challenges Int J Med Inform 2003; 71(2-3 ): 125-35.
[25] Westbrook JI, Braithwaite J, Georgiou A. Multimethod evaluation of information and communication technologies in health in the context of wicked problems and sociotechnical theory J Am Med Inform Assoc 2007; 14(6 ): 746-55.
[26] Stoop AP, Berg M. Integrating quantitative and qualitative methods in patient care information system evaluation: guidance for the organizational decision maker Methods Inform Med 2003; 42(4 ): 458-62.
[27] Scott P, Briggs J. A pragmatist argument for mixed methodology in medical informatics J Mixed Methods Res 2009; 3(3 ): 223-0.
[28] Kaplan B, Shaw N. Future directions in evaluation research: people, organizational, and social issues Methods Inform Med 2004; 43(3 ): 215-31.
[29] Pasmore W. Social science transformed: the socio-technical perspective Hum Relat 1995; 48(1 ): 1-21.
[30] Callen J, Braithwaite J, Westbrook J. Cultures in hospitals and their influence on attitudes to, and satisfaction with, the use of clinical information systems Soc Sci Med 2007; 65(3 ): 635-9.
[31] Aarts J, Berg M. Same systems, different outcomes--comparing the implementation of computerized physician order entry in two Dutch hospitals Methods Inform Med 2006; 45(1 ): 53-61.
[32] Niazkhani Z, van der Sijs H, Pirnejad H, et al. Same system, different outcomes: comparing the transitions from two paper-based systems to the same computerized physician order entry system Int J Med Inform 2009; 78(3 ): 170-81.
[33] Peute L, Aarts J, Bakker P, Jaspers M. Anatomy of a failure: A sociotechnical evaluation of a laboratory physician order entry system implementation Int J Med Inform 2010; 74(4 ): 58-70.
[34] Norman D. Emotional Design: Why we love (or hate) everyday things 2004.
[35] Greatbatch D, Murphy E, Dingwall R. Evaluating medical information systems: ethnomethodological and interactionist approaches Health Serv Manage Res 2001; 14(3 ): 181-91.
[36] Heng HKS, McGeorge WD, Loosemore M. Beyond strategy: Exploring the brokerage role of facilities manager in hospitals J Health Org Manag 2005; 19: 16-31.
[37] Ouwens M, Hulscher M, Akkermans R, Hermens R, Grol R, Wollersheim H. The team climate inventory: application in hospital teams and methodological considerations Qual Saf Health Care 2008; 17(4 ): 275-0.
[38] Georgiou A, Westbrook J, Braithwaite J, et al. When requests become orders—A formative investigation into the impact of a computerized physician order entry system on a pathology laboratory service Int J Med Inform 2007; 76(8 ): 583-91.
[39] Boyce NW, Boyce NW. Potential pitfalls of healthcare performance indicators Med J Aust 2002; 177(5 ): 229-30.
[40] Pawson R. A measure for measures: a manifesto for empirical sociology 1989.
[41] Woods D. Designs are hypotheses about how artifacts shape cognition and collaboration Ergonomics 1998; 41(2 ): 168-73.
[42] Nykänen P, Brender J, Ammenwerth E. Guidelines for Good Evaluation Practices in Health Informatics (v0.16) 2008 [cited 2009 Jan 5]: Available from: http://iig.umit.at/efmi/gep-hi/GEP_HI_v016_Dec012008.doc.
[43] Cusack C, Byrne C, Hook J, McGowan J, Poon E, Zafar A. Health information technology evaluation toolkit: 2009 Update. Rockville, MD: Agency for Healthcare Research and Quality Report No.: AHRQ Publication No. 09-0083-EF. June 2009 2009.
[44] Berg M. Patient care information systems and health care work: a sociotechnical approach Int J Med Inform 1999; 55(2 ): 87-101.

Endorsements



"Open access will revolutionize 21st century knowledge work and accelerate the diffusion of ideas and evidence that support just in time learning and the evolution of thinking in a number of disciplines."


Daniel Pesut
(Indiana University School of Nursing, USA)

"It is important that students and researchers from all over the world can have easy access to relevant, high-standard and timely scientific information. This is exactly what Open Access Journals provide and this is the reason why I support this endeavor."


Jacques Descotes
(Centre Antipoison-Centre de Pharmacovigilance, France)

"Publishing research articles is the key for future scientific progress. Open Access publishing is therefore of utmost importance for wider dissemination of information, and will help serving the best interest of the scientific community."


Patrice Talaga
(UCB S.A., Belgium)

"Open access journals are a novel concept in the medical literature. They offer accessible information to a wide variety of individuals, including physicians, medical students, clinical investigators, and the general public. They are an outstanding source of medical and scientific information."


Jeffrey M. Weinberg
(St. Luke's-Roosevelt Hospital Center, USA)

"Open access journals are extremely useful for graduate students, investigators and all other interested persons to read important scientific articles and subscribe scientific journals. Indeed, the research articles span a wide range of area and of high quality. This is specially a must for researchers belonging to institutions with limited library facility and funding to subscribe scientific journals."


Debomoy K. Lahiri
(Indiana University School of Medicine, USA)

"Open access journals represent a major break-through in publishing. They provide easy access to the latest research on a wide variety of issues. Relevant and timely articles are made available in a fraction of the time taken by more conventional publishers. Articles are of uniformly high quality and written by the world's leading authorities."


Robert Looney
(Naval Postgraduate School, USA)

"Open access journals have transformed the way scientific data is published and disseminated: particularly, whilst ensuring a high quality standard and transparency in the editorial process, they have increased the access to the scientific literature by those researchers that have limited library support or that are working on small budgets."


Richard Reithinger
(Westat, USA)

"Not only do open access journals greatly improve the access to high quality information for scientists in the developing world, it also provides extra exposure for our papers."


J. Ferwerda
(University of Oxford, UK)

"Open Access 'Chemistry' Journals allow the dissemination of knowledge at your finger tips without paying for the scientific content."


Sean L. Kitson
(Almac Sciences, Northern Ireland)

"In principle, all scientific journals should have open access, as should be science itself. Open access journals are very helpful for students, researchers and the general public including people from institutions which do not have library or cannot afford to subscribe scientific journals. The articles are high standard and cover a wide area."


Hubert Wolterbeek
(Delft University of Technology, The Netherlands)

"The widest possible diffusion of information is critical for the advancement of science. In this perspective, open access journals are instrumental in fostering researches and achievements."


Alessandro Laviano
(Sapienza - University of Rome, Italy)

"Open access journals are very useful for all scientists as they can have quick information in the different fields of science."


Philippe Hernigou
(Paris University, France)

"There are many scientists who can not afford the rather expensive subscriptions to scientific journals. Open access journals offer a good alternative for free access to good quality scientific information."


Fidel Toldrá
(Instituto de Agroquimica y Tecnologia de Alimentos, Spain)

"Open access journals have become a fundamental tool for students, researchers, patients and the general public. Many people from institutions which do not have library or cannot afford to subscribe scientific journals benefit of them on a daily basis. The articles are among the best and cover most scientific areas."


M. Bendandi
(University Clinic of Navarre, Spain)

"These journals provide researchers with a platform for rapid, open access scientific communication. The articles are of high quality and broad scope."


Peter Chiba
(University of Vienna, Austria)

"Open access journals are probably one of the most important contributions to promote and diffuse science worldwide."


Jaime Sampaio
(University of Trás-os-Montes e Alto Douro, Portugal)

"Open access journals make up a new and rather revolutionary way to scientific publication. This option opens several quite interesting possibilities to disseminate openly and freely new knowledge and even to facilitate interpersonal communication among scientists."


Eduardo A. Castro
(INIFTA, Argentina)

"Open access journals are freely available online throughout the world, for you to read, download, copy, distribute, and use. The articles published in the open access journals are high quality and cover a wide range of fields."


Kenji Hashimoto
(Chiba University, Japan)

"Open Access journals offer an innovative and efficient way of publication for academics and professionals in a wide range of disciplines. The papers published are of high quality after rigorous peer review and they are Indexed in: major international databases. I read Open Access journals to keep abreast of the recent development in my field of study."


Daniel Shek
(Chinese University of Hong Kong, Hong Kong)

"It is a modern trend for publishers to establish open access journals. Researchers, faculty members, and students will be greatly benefited by the new journals of Bentham Science Publishers Ltd. in this category."


Jih Ru Hwu
(National Central University, Taiwan)


Browse Contents



Webmaster Contact: info@benthamopen.net
Copyright © 2019 Bentham Open