The Centre for Addiction and Mental Health (CAMH) is a 500-bed freestanding mental health hospital with 3000 employees, including 130 full-time medical staff. There are a total of eight Clinical Programs or service lines, and an active Primary Care service. We currently have a hybrid of electronic and paper documentation systems; physician orders are on paper with the exception of laboratory and dietary orders, which are electronic. Our order set (OS) initiative was launched to aid our preparations for a Computerized Provider Order Entry (CPOE) project that was imminent. To assist with this project a vendor was used to support the development of order sets (OSs) and the corresponding clinical process redesign work that are mandatory prerequisites to CPOE. The goal was to develop and implement OSs in a hybrid clinical information environment across eight inpatient programs over one year. Reasons for OSs are many [1Bobb AM, Payne TH, Gross PA. Viewpoint: controversies surrounding use of order sets for clinical decision support in computerized provider order entry J Am Med Inform Assoc 2007; 14: 41-7.], including 1) OSs are required as clinical content in preparation for Computerized Provider Order Entry (CPOE) [2Ash JS, Stavri PZ, Kuperman GJ. A consensus statement on considerations for a successful CPOE Implementation J Am Med Inform Assoc 2003; 10: 229-34.,3Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system J Am Med Inform Assoc 2003; 10: 322-9.] ; 2) Paper-based OSs will ready our staff for CPOE and standardization of care [4Heffner JE, Brown K, Ellis R, Brown S. Using intranet-based order sets to standardize clinical care and prepare for computerized physician order entry Jt Comm J Qual Saf 2004; 30: 366-76.] ; and 3) OSs can improve quality [5Ozdas A, Speroff T, Waitman R, Ozbolt J, Butler J, Miller RA. Integrating “best of care” protocols into clinicians' workflow via care provider order entry: impact on quality-of-care indicators for acute myocardial infarct J Am Med Inform Assoc 2006; 13: 188-96.-8Schnipper JL, Liang CL, Ndumele CD, Pendergrass ML. Effects of a computerized order set on inpatient management of hyperglycemia: a cluster-randomized trial Endocr Pract 2010; 16: 209-18.], safety [9Hermayer KL, Cawley P, Arnold P, et al. Impact of improvement efforts on glycemic control and hypoglycemia at a university medical center J Hosp Med 2009; 4: 331-9.], resource utilization [10Guttmann A, Zagorski B, Austin PC, et al. Effectivness of Emergency Department Asthma Management Strategies on Return Visits in Children: A Population-Based Study Pedaitrics 2007; 120: e1402-0.], order completeness, workflow, staff and patient satisfaction [11Institute for Safe Medication Practices. ISMP's Guidelines for Standard Order Sets. PA, USA: ISMP 2010; pp. 1-10.]. As is the case with the use of electronic systems more generally, OSs are widely used in physical medicine, but less so in behavioral health. As inpatient psychiatry gradually adopts electronic medical records, OSs will play a growing role in bringing clinical content to electronic ordering. Our main research questions were: How readily will clinicians adopt paper order sets in a hybrid information system? What obstacles will need to be overcome?

We applied a variant of the methodology recommended by our vendor, PatientOrderSets.com. In February an OS working group (OSWG) was formed and it consisted of representatives from physicians, nursing informatics, clinical forms administration, medical informatics, health information systems analyst, allied health, nursing (nurse educator and advanced practice nurse), health records, laboratory, and clinical informatics. The OSWG began having weekly meetings in March 2010. Its mandate included:

1. support the creation and maintenance of standardized modular OSs;
2. establish a Standard Reference OS with the oversight of PatientOrderSets.com for a guideline in the creation of CAMH OSs;
3. facilitate multidisciplinary communication and collaboration;
4. ensure compliance with Institute and Safe Medication Practices (ISMP) standards, hospital abbreviation standards, hospital formulary, and any hospital policies that may influence OS content.

Meetings were set with all physicians of each clinical program and with managers and their clinical staff to inform them of the implementation plan, educate them about OSs and convey what was expected of them.

The eight clinical inpatient programs and primary care were informed that members from each program were required to participate on their local program content teams in April. There was an initial meeting with each content team in May to inform them of the time line (6 weeks) to develop their OSs complete with input and local consensus from all physicians in their program [12Novak LL. Making sense of clinical practice: order set design strategies in CPOE AMIA Annu Symp Proc 2007; 2007: 568-72.]. Each team would meet weekly there after to develop content for their respective programs; they were also responsible to seek feedback and agreement from all clinicians, including review by Clinical Program Medical Directors. Throughout the development phase, the OSWG would also meet weekly to review and recommend changes to the originating content team.

After final revisions were completed on the OSs, they were submitted for approval. The first level of approval was from the OSWG; OSs then went on to the Pharmacy and Therapeutic Committee for subsequent approval; then lastly to the Medical Advisory Committee (MAC) for final vetting. Table 1 shows all the OSs that were finally approved by the MAC.

Table 1

Order Sets with Corresponding Content Teams

The paper OSs consisted of physician orders, alerts and clinical information. After final approval at MAC, OSs were implemented. Communications were sent to all clinicians outlining instructions, implementation dates, available education, support and contact information. The medical staff received training in the use of OS at their usually scheduled physician meetings and were supplied all relevant implementation information. Throughout the implementation of the emergency department all residents on their ED rotation were educated in the use of the paper OSs. Clinical staff provided post-implementation comments and suggestions to the lead of the OSWG.

Implementation was over three months starting in late September and finishing mid-December.

During the implementation of the twenty-two OSs, clinical support was provided to each area in the form of a clinical informatics nurse physically present for 12 hours daily, and then being available via phone for the remaining 12 hrs. Initial support lasted one to three weeks depending on the size of the program. Following the initial support period, 24-hour phone support by the clinical informatics nurse went for a period of 6 months (Table 2).

Table 2

Order Sets Implementation Schedule

This project was approved by our institutional research ethics board. In order to help assure support and engagement at all levels of the organization there was regular communication with the executive team which started in January 2010. Communications and updates to all clinicians were ongoing for the project's duration.

The clinical programs employed OSs and the direct admission OSs were added to each unit's routine paper admission packet. Auditing of admission OS use was done six months post-implementation and we observed that 150/152 (98.7%) of all admissions in March and April 2011 used a Direct Admission or General Admission via the ED OS. Sixty percent of audited admissions used the General Admission via the Emergency Department (ED) OS, as the ED is the entry route for most CAMH inpatients. Nursing signatures were complete in 146/150 (97.3%) of admissions. Post-implementation reports from nursing indicated that nurses were very much in favour of the paper-based OSs for their completeness, clarity, and efficiency in providing care. At the same time, reports from the Emergency Department (ED) informed the OSWG that their admission OS was too lengthy.

Throughout the development and implementation stages communication with and engagement of clinicians was essential to the success of the project. Post-implementation reports from nursing indicated that nurses were very much in favour of the paper-based OSs for their completeness, clarity, and efficiency in providing care.

Several issues were encountered during implementation and were responded to with iterative modifications to order sets. Changes requested by clinical areas were made, revised OSs were approved, put into clinical use, and that cycle was repeated as necessary. Nursing highlighted one key area: nurses questioned why they were required to sign off items on an OS that were not doctors' orders. Sign off of such items was required in some OSs because some of the content teams had tried to use the OSs to improve care quality by including nursing tasks that do not require a physician order, or by including certain kinds of documentation (e.g. alerts). The OSs have since been reviewed and unnecessary orders, fields and items have been removed. A second issue was the implementation of paper OSs in a hybrid paper-electronic set of clinical systems. Hybrid OSs were needed, with the majority being on paper, while lab and dietary orders were electronic. Specific groups of lab orders were created, one for each OS, within our electronic ordering system. The splitting of OSs into digital and analog portions made necessary by our hybrid environment reduced the usability of OSs. An accompanying benefit was a rise in the rate of electronic lab orders at admission. Thirdly, most inpatient areas introduced OSs and a separate and new inpatient medication reconciliation procedure at the same time, so staff had to accommodate two process changes.

As noted above, another problem requiring resolution happened in the Emergency Department (ED). Our ED is a high-volume area staffed mainly by a large number of residents who rotate, with any given resident providing periodic coverage. Medical staff and residents in the ED found that the admission OS was too long – combined with the new admission medication reconciliation steps, ED admission ordering became too complex. Both medication reconciliation and OS projects had an opportunity to simplify their processes: the CATS (ED) content team reduced the General Admission via the ED OS from three pages to one. The revised abbreviated version was accepted by medical staff and residents in the ED and is currently in use.

There were some limitations to the OS design and implementation. Usability testing [13Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets? J Am Med Inform Assoc 2011; 18: 276-81., 14Chan J, Shojania KG, Easty AC, Etchells EE. Usability evaluation of order sets in a computerized provider order entry system BMJ Qual Saf 2011; 20(11): 932-40.] was not part of the project, and would have helped to avoid the flow problem in the ED. For speed and ease of implementation with limited resources, we focused on the pragmatic local consensus method [12Novak LL. Making sense of clinical practice: order set design strategies in CPOE AMIA Annu Symp Proc 2007; 2007: 568-72.], based on our standard of care. Lastly, the foundation of all CAMH OSs was a standard reference OS based on the vendor's standard reference OS derived from acute care settings; as such the standard reference OS was not behavioral health-specific.

Over one year, we established an approvals process, developed OS content, and implemented twenty-two inpatient OSs in eight clinical programs. To our knowledge, this is the first report on order sets from a freestanding psychiatric facility with a hybrid information environment. Reflecting our hybrid clinical systems, the OSs themselves were hybrid, principally paper-based, with corresponding electronic dietary and laboratory portions for each OS. Initially the OSs were designed to be quite comprehensive; experience of the hybrid OSs by ED and nursing staff has since led to review, with scrutiny to augment efficiency. Despite a hybrid information environment, and a separate concurrent medication reconciliation project, clinical staff fully embraced OSs for inpatient behavioral health. In the future, we will increase evidence-based content, convert the OSs to fully electronic versions for CPOE, and add mental health protocols and pathways.

The author confirms that this article content has no conflict of interest.