In many major medical centers, overnight radiology exams are initially interpreted by an on-call radiology resident and reviewed the following morning by a staff radiologist. It is a common perception that resident interpretations result in more errors as compared to staff radiologists. Multiple prior studies have been performed to examine this perception [1-3]. The goal of these studies has been to determine whether radiology resident autonomy during overnight hours is a safe practice for patient care or if increased hours of staff radiologist coverage should be implemented. However, few studies have examined the impact of increasing exam volumes on discrepancy rates in primary interpretations performed by on-call residents, particularly with respect to neuroradiology CT exams.

The purpose of our study is to determine whether there is a correlation between increasing neuroradiology CT exam volumes and major discrepancy rates in primary interpre-tations performed by on-call residents at our institution.

A retrospective review was made of all on-call neuroradiology CT exams interpreted between the hours of 2300 and 0800 at our institution over a 5-year period from January 1, 2006 through December 31, 2010. All neuroradiology CT exams performed during these hours are initially interpreted by an on-call resident and reviewed by a staff neuroradiologist the following morning. These include CT exams of the head, sinuses, neck, and spine, including CT angiography (CTA) of these regions, in both adult and pediatric patients. The on-call resident was typically a 3^rd or 4^th year resident and rarely a 2^nd year resident (less than 5%).

Our institution is a busy multispecialty center covering a single emergency room. The on-call resident is responsible for interpreting body and neurologic CT exams, x-rays, and ultrasound. MRI exams are not covered by the on-call resident.

At our institution, the on-call resident dictates a final report at the time of initial interpretation. A staff neuroradiologist reviews this report the following morning and determines if there are any discrepancies between the resident’s report and the staff’s interpretation of the findings. If there are no discrepancies, the staff neuroradiologist signs off on the resident’s report without any changes. If there is a discrepancy, the staff neuroradiologist determines whether to categorize it as “minor” or “major”. A minor discrepancy is an error that the staff neuroradiologist deems not significant enough to directly impact patient care. In the event of a minor discrepancy, the staff neuroradiologist changes the resident’s report and an automatic email is generated to inform the ordering clinician of the modification to the report. A major discrepancy is an error that the staff neuroradiologist deems significant enough to directly impact patient care. In the event of a major discrepancy, the staff neuroradiologist not only changes the resident’s report, but also speaks directly to the ordering clinician or service and documents this communication in the modified report. The on-call resident receives feedback in the form of an automatically generated email if a major or minor discrepancy occurs in addition to direct feedback at the discretion of the supervising staff.

The number of major discrepancies and the total number of neuroradiology CT exams interpreted by on-call residents during the hours defined above were recorded by quarter for each year in the study time period. The major discrepancy rate was defined as the ratio of major discrepancies to the total number of neuroradiology CT examinations interpreted. Due to generally higher on-call volumes on weekend nights, the major discrepancy rate was calculated separately for exams performed on weeknights (Monday through Thursday) and those performed on weekend nights (Friday through Sunday).

Over the time frame of our study, an increased workload was noted during both the weekday and weekend volumes for neuroradiology exams.

Weekday Volumes

The average number of on-call neuroradiology CT exams performed on weeknights increased from 3.0 studies per night in 2006 to 5.2 studies per night in 2010. A relatively steady trend of gradual increase was clearly seen in the average number of studies interpreted per night. A similar trend was also observed in the maximum number of exams interpreted on a weeknight with an increase from an average of 8.2 studies per night in 2006 to 11.0 studies per night in 2010 (Fig. 1). Both of these increases were statistically significant with p<0.001.

Fig. (1). Weeknight Neuroradiology CT Volumes.

Fig. (2). Weekend Neuroradiology CT Volumes.

Fig. (3). Major discrepancy rate.

Our study demonstrates that despite an increase in on-call neuroradiology CT exam volumes, the major discrepancy rate in primary interpretations performed by on-call residents at our institution has remained stable and low. These findings are important because there is a perception that resident autonomy, particularly in a busy, high volume radiology practice, leads to increased significant errors. Our study shows that increased volume by itself does not necessarily correlate with an increased major discrepancy rate.

Multiple previous studies have analyzed errors in primary interpretations of neuroradiology CT exams performed by on-call residents. Most of these studies have specifically analyzed head CT. These studies have shown similar low major discrepancy rates to our study [4,5]. One study noted a decreasing major discrepancy rate with increasing resident seniority [6]. Prior studies have also shown that major discrepancy rates among subspecialty neuroradiologist interpretations are similar to those seen in studies of on-call radiology residents [7,8]. However, to our knowledge, there has not been a previous study that has analyzed the impact of increasing on-call neuroradiology CT exam volumes on the major discrepancy rate. Our study adds to the current literature by establishing that increased exam volumes do not necessarily correlate with an increased major discrepancy rate.

Resident autonomy is an integral part of radiology education. In our residency training program, much of the autonomous resident experience is acquired during on-call periods. These shifts are typically higher volume than daytime shifts and allow residents to learn how to efficiently manage a busy, multimodality workload. As radiology practices continue to become busier, it has been suggested that continued expansion of off-hours staff radiologist coverage should be implemented to decrease error rates [9].

While there may be legitimate arguments for increased staff coverage, our study suggests that increasing exam volumes and the fear of increased major discrepancies should not be a substantial basis for that argument. In addition, eliminating the independence of residents during after-hours periods would significantly detract from their educational experience.

Our study does have several limitations. This study only evaluated neuroradiology CT exams. The on-call residents were also interpreting exams from other modalities and subspecialties during their shifts, and the impact of this on the overall exam volume during a particular shift was not taken into account. Also, data was analyzed and averaged on a quarterly basis over the study period. This was useful for demonstrating general trends but was not detailed enough to evaluate whether major discrepancies occurred during particularly high volume shifts or high volume periods within a shift. A study of on-call resident interpretations of cross-sectional imaging across multiple modalities and subspecialties suggested that increased cases per shift did lead to a higher, although still low, major discrepancy rate [10]. Another limitation of our study, and all studies that examine major discrepancy rate, is the inherent subjective nature of a major discrepancy. Because this is not an objective measure, there may be significant variation in staff neuroradiologist opinions on whether a particular error is categorized as a major or minor discrepancy. No data was available with multiple staff neuroradiologists reviewing each error to demonstrate whether there was consensus about which errors represented major discrepancies. Perhaps a better metric to evaluate errors would be adverse patient outcomes, and this was not addressed by our study. Previous studies, however, have shown a low incidence of adverse patient outcomes in the setting of major discrepancies [11, 12]. It is also possible that the results observed in our study were due to overall improvement in on-call resident performance, although no changes were made to the training program or the on-call resident duties during the study.

While our study demonstrates important results, it also presents opportunities for further investigation as outlined by the aforementioned limitations. Each major discrepancy could be analyzed in greater detail to determine whether it occurred during a particularly busy shift or particularly busy time during a shift (multiple traumas, etc.) and if there was an adverse effect on patient outcome. The subjective nature of a major discrepancy could also be analyzed. Blinded review of cases with discrepancies by multiple staff radiologists could be performed to determine concordance between categorization of discrepancies as major or minor. A previous study suggests that interobserver agreement is poor among staff radiologists when grading severity of discrepancies across multiple modalities and specialties [13]. Defining a major discrepancy in an objective manner would also be helpful in more accurately categorizing radiology errors, although the intrinsically subjective nature of a major discrepancy makes this difficult. Follow-up with clinicians to determine which errors actually had an impact on patient care and to what degree would also be instructive, although the time consuming nature of this process would likely limit its feasibility.

Despite an increasing number of on-call neuroradiology CT exams at our institution, the major discrepancy rate in primary interpretations performed by residents has remained stable and low. Resident autonomy should not be limited solely due to increasing on-call exam volumes.