Variable use of phrases expressing diagnostic uncertainty can lead to ambiguous radiology reports, a concern for information processing.
This study aimed to quantify the usage of phrases conveying diagnostic certainty for abdominal imaging findings and assess factors that impact use of phrases with “good agreement” between radiologists and referring providers.
This retrospective, Institutional Review Board-Approved study included all diagnostic reports generated by the Abdominal Radiology Division at an academic medical center July-September 2016. We assessed the use of 16 diagnostic certainty phrases using information retrieval from the Impression section of radiology reports. Phrases with good provider agreement for conveying the level of certainty are defined as “good agreement” phrases - including “diagnostic of”, “represents” and “unlikely.” We assessed the impact of imaging modality, trainee contribution to report generation, and individual radiologists.
In 5,598 radiology reports, 2,071 (37%) contained diagnostic certainty phrases, 119 (6%) of which were “good agreement” phrases. There was a significant difference between how frequently “good agreement” phrases were used in Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and X-ray reports (p=0.0003). There was a significant variation among attending radiologists on the use of “good agreement” phrases (p<0.0019). There was no difference in the use of “good agreement” phrases in reports generated by attending radiologists alone compared to reports with trainees.
Although phrases to convey diagnostic certainty were commonly used in radiology reports, the use of phrases with a good agreement was uncommon. Standardizing terminology to convey diagnostic certainty may reduce ambiguity in radiology reports and generate more accurate information processing tools.
Open Peer Review Details | |||
---|---|---|---|
Manuscript submitted on 2-9-2018 |
Original Manuscript | Factors Impacting the Use of Terminology to Convey Diagnostic Certainty in Radiology Reports |
Radiology reports serve as the primary method of communication between radiologists and other healthcare providers [1Friedman PJ. Radiologic reporting: Structure. AJR Am J Roentgenol 1983; 140(1): 171-2.
[http://dx.doi.org/10.2214/ajr.140.1.171] [PMID: 6600314] -3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] ]. Many studies and interventions have been implemented to increase timely report generation, decrease error rates in reporting, and provide comprehensive documentation of findings [4Augmenting the impact of technology adoption with financial incentive to improve radiology report signature times. Journal of the American College of Radiology: JACR 2010; 7(3): 198-204.-10Prevedello LM, Farkas C, Ip IK, et al. Large-scale automated assessment of radiologist adherence to the Physician Quality Reporting System for stroke. J Am Coll Radiol 2012; 9(6): 414-20.
[http://dx.doi.org/10.1016/j.jacr.2012.01.014] [PMID: 22632668] ]. Information extraction and natural language processing tools have integrated certainty modifiers when extracting disease concepts [11Krauthammer M, Hripcsak G. knowledge model for the interpretation and visualization of NLP-parsed discharged summaries. 2001; In: Proceedings / AMIA Annual Symposium AMIA Symposium; 2001; pp. 339-43.]. However, despite multiple studies reporting radiologists' usage of vague and uncertain phrases in their reports [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] , 12Chapman WW, Fiszman M, Frederick PR, Chapman BE, Haug PJ. Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia. Acad Radiol 2001; 8(1): 57-66.
[http://dx.doi.org/10.1016/S1076-6332(03)80744-4] [PMID: 11201458] -14Hobby JL, Tom BD, Todd C, Bearcroft PW, Dixon AK. Communication of doubt and certainty in radiological reports. Br J Radiol 2000; 73(873): 999-1001.
[http://dx.doi.org/10.1259/bjr.73.873.11064655] [PMID: 11064655] ], there has been little effort to evaluate the use of terminology conveying diagnostic certainty in radiology reports, and whether providers agree on the level of certainty they convey.
While ideally, radiology reports are concise and unambiguous [15Berlin L. Communicating findings of radiologic examinations: Whither goest the radiologist’s duty? AJR Am J Roentgenol 2002; 178(4): 809-15.
[http://dx.doi.org/10.2214/ajr.178.4.1780809] [PMID: 11906852] , 16Berlin L. Pitfalls of the vague radiology report. AJR Am J Roentgenol 2000; 174(6): 1511-8.
[http://dx.doi.org/10.2214/ajr.174.6.1741511] [PMID: 10845472] ], in practice, there is wide variability in the terminology used by radiologists in reporting findings, depending on their degree of certainty and judgment [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] , 9Langlotz CP, Meininger L. Enhancing the expressiveness and usability of structured image reporting systems. 2000; In: Enhancing the expressiveness and usability of structured image reporting systems. Proceedings / AMIA Annual Symposium AMIA Symposium.; 2000; pp. 467-71.]. This variability in expressing certainty has been significantly associated with reports that are more ambiguous [12Chapman WW, Fiszman M, Frederick PR, Chapman BE, Haug PJ. Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia. Acad Radiol 2001; 8(1): 57-66.
[http://dx.doi.org/10.1016/S1076-6332(03)80744-4] [PMID: 11201458] ]. Standardized reporting has been shown to decrease ambiguity for particular radiology procedures [17van Riel SJ, Ciompi F, Jacobs C, et al. Malignancy risk estimation of screen-detected nodules at baseline CT: Comparison of the PanCan model, Lung-RADS and NCCN guidelines. Eur Radiol 2017; 27(10): 4019-29.
[http://dx.doi.org/10.1007/s00330-017-4767-2] [PMID: 28293773] -20Feig SA. Role and evaluation of mammography and other imaging methods for breast cancer detection, diagnosis, and staging. Semin Nucl Med 1999; 29(1): 3-15.
[http://dx.doi.org/10.1016/S0001-2998(99)80026-9] [PMID: 9990680] ]. Breast Imaging Reporting and Data System [BI-RADS] for breast imaging reporting has been utilized to help understand and communicate the probability of a pathologic finding [19Müller-Schimpfle M, Wersebe A, Xydeas T, et al. Microcalcifications of the breast: How does radiologic classification correlate with histology? Acta Radiol 2005; 46(8): 774-81.
[http://dx.doi.org/10.1080/02841850500270274] [PMID: 16392601] ]. Similarly, Prostate Imaging Reporting and Data System (PI-RADS) and Lung Reporting and Data System (Lung-RADS) have been used to communicate the probability of prostate and lung cancer, respectively [17van Riel SJ, Ciompi F, Jacobs C, et al. Malignancy risk estimation of screen-detected nodules at baseline CT: Comparison of the PanCan model, Lung-RADS and NCCN guidelines. Eur Radiol 2017; 27(10): 4019-29.
[http://dx.doi.org/10.1007/s00330-017-4767-2] [PMID: 28293773] , 18Rosenkrantz AB, Oto A, Turkbey B, Westphalen AC. Prostate imaging reporting and data system (PI-RADS), version 2: A critical look. AJR Am J Roentgenol 2016; 206(6): 1179-83.
[http://dx.doi.org/10.2214/AJR.15.15765] [PMID: 26913638] ]. However, expressing the probability of other imaging findings, other than these few malignancies, is a difficult task. Moreover, the standardized terminology for conveying diagnostic certainty in abdominal radiology reports is not yet established.
A previous study identified 15 commonly-used phrases utilized by radiologists for expressing certainty [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] ]. In that study, the two phrases which had the highest agreement among radiologists in conveying the level certainty were “diagnostic of” and “unlikely”, based on a questionnaire wherein radiologists voluntarily ranked 15 phrases in order of diagnostic certainty. The phrase “diagnostic for” and “represents” were ranked with the highest likelihood in conveying malignancy by both radiologists and primary care physicians in a separate study [13Gunn AJ, Tuttle MC, Flores EJ, et al. Differing interpretations of report terminology between primary care physicians and radiologists. J Amer College Radiol: JACR 2016; 13(12 Pt A): 1525-9.]. The purpose of this study was to assess the usage of 16 phrases to convey diagnostic certainty for imaging findings in the Abdominal Division of an academic radiology practice [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] , 12Chapman WW, Fiszman M, Frederick PR, Chapman BE, Haug PJ. Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia. Acad Radiol 2001; 8(1): 57-66.
[http://dx.doi.org/10.1016/S1076-6332(03)80744-4] [PMID: 11201458] ], specifically to quantify the usage of phrases with “good agreement” between radiologists and referring providers that are expected to express less ambiguity. Understanding the level of variability and factors that impacted the usage of phrases with “good agreement” will help in developing standard terminology to convey diagnostic certainty.
This retrospective, Health Insurance Portability and Accountability Act-compliant study was approved by the Institutional Review Board, and performed at an urban, quaternary care, academic medical center. All abdominal X-ray, Computed Tomography (CT) scan and Magnetic Resonance Imaging (MRI) radiology reports generated by the Abdominal Radiology Division in the third quarter of 2016 (July-September 2016) were included in the analysis. Exclusion criteria included reports from other modalities (e.g., Upper Gastro-Intestinal series, Hysterosalpingogram) and abdominal interventional procedures (e.g., paracentesis). The radiology department has 13 abdominal radiology subspecialty-trained attending physicians, 5 abdominal radiology subspecialty fellows and 39 diagnostic radiology residents. Every finalized radiology report is signed by one of the attending physicians, whether or not it was first reviewed by a trainee.
We assessed radiology reports for the use of 16 previously-identified phrases that convey diagnostic certainty [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] ]. In particular, we evaluated the use of three phrases for which radiologists had “good agreement” with referring providers in conveying diagnostic certainty - “diagnostic of/for”, “represents” and “unlikely” - as well as 13 phrases for which there was variable agreement among radiologists and non-radiologists about conveying diagnostic certainty [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] ].
We utilized information retrieval and query matching to extract phrases conveying diagnostic certainty from within the impression sections of radiology reports. We utilized a publicly available toolkit, Information from Searching Content with an Ontology-Utilizing Toolkit (iSCOUT) for this task [21Lacson R, Andriole KP, Prevedello LM, Khorasani R. Information from Searching Content with an Ontology-Utilizing Toolkit (iSCOUT). J Digit Imaging 2012; 25(4): 512-9.
[http://dx.doi.org/10.1007/s10278-012-9463-9] [PMID: 22349993] ]. The Header Extractor component in iSCOUT enables selective retrieval of sections of reports by restricting a search to content within various headers of a report (e.g., Impression, Findings). From the Impression section, we subsequently performed query matching for each individual diagnostic certainty phrase using Microsoft Access (Microsoft Corporation, Redmond, Washington). Reports that included at least one of the 16 phrases conveying diagnostic certainty were retrieved.
We assessed the proportion of reports with phrases conveying diagnostic certainty, as well as the frequency and distribution of those phrases. In addition, we calculated the proportion of reports with “good agreement” phrases out of all reports with phrases that convey diagnostic certainty. Each report that contains one or more phrases that convey diagnostic certainty is counted once in the denominator. Each report is counted once in the numerator (i.e. reports with “good agreement” phrases) if it contains at least one “good agreement” phrase. We assessed the impact of imaging modality, trainee contribution to report generation and individual radiologists in the usage of “good agreement” phrases. Chi-square test was performed to evaluate the difference between the proportion of reports with “good agreement” phrases and reports from X-ray, CT scan and MRI examinations, between attending-only reports or joint attending and trainee reports, and between the individual attendings. For comparing individual attendings, those who contributed fewer than 100 reports were excluded from statistical analysis. All statistical analyses were performed using commercially available software (R 3.4.1 software, Vienna, Austria). A p-value of <0.05 was considered statistically significant.
During the study period, a total of 5,598 unique radiology reports were generated by the Abdominal Radiology Section; 1,226 (22%) abdominal X-ray reports, 3,010 (54%) CT scan reports and 1,362 (24%) MRI reports. There were 1,734 attending-only reports and 3,864 joint attending and trainee reports. The 13 radiologist attendings accounted for an average of 430 reports each. Among 5,598 reports, 2,071 (37%) contained at least one diagnostic certainty phrase.
Fig. (1) demonstrates the proportion of reports containing phrases used to convey diagnostic certainty out of all reports, by imaging modality (examples of phrase usage are provided in Table 1. Only 119 (6%) of phrases were “good agreement” phrases. The most commonly used diagnostic certainty phrases in abdominal imaging were “likely” (including with modifiers), and “may represent”, accounting for 15% and 5%, respectively, of all reports with phrases that convey diagnostic certainty. Specifically, in abdominal x-ray reports, the three most common phrases were “likely”, “may represent”, and “consistent with”; in CT scans, the most common words were “likely”, “may represent” and “may be”; in MRI imaging, the most frequently used phrases were “likely”, “likely (with modifiers)”, and “consistent with”.
Fig. (1) Proportion of reports containing phrases (n=16) used to convey diagnostic certainty out of all reports, by modality. |
Table 2 shows the results of bivariate analyses comparing use of phrases with “good agreement” between imaging modalities. There was significant difference in how frequently phrases with “good agreement” were used as compared with phrases with variable agreement between imaging modalities. Reports from MRI and CT exams had a higher usage of “good agreement” phrases compared to X-ray reports (p=0.0003; chi-square analysis).
Table 3 shows the results of bivariate analyses comparing use of phrases with “good agreement” between reports generated by attending physicians, and trainees with attending physicians. No significant difference was noted in usage of phrases with “good agreement” between these two groups (p<0.5169). Fig. (2) shows the proportion of reports containing phrases used to convey diagnostic certainty out of all reports generated by attending-only and trainees with attending physicians. Reports that were generated by attending physicians had “likely”, “may be” and “may represent” as most commonly used phrases, whereas joint reports from both attending and trainee physicians most commonly contained “likely”, “may represent” and “consistent with”.
Fig. (2) Proportion of reports containing phrases (n=16) used to convey diagnostic certainty in attending-only and trainees with attending physician reports. |
Individual attending physicians used diagnostic certainty phrases with “good agreement” in 0-14% of their reports that contain diagnostic certainty phrases. Table 4 demonstrates a significant difference in use of phrases with “good agreement” in reports generated between individual attending physicians (p<0.0019).
In a previous study, surveyed radiologists and non-radiologists ranked the phrases “diagnostic of” and “represents” as the phrases which conveyed the highest degree of certainty for expressing diagnostic findings in radiology reports [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] ]. Phrases such as “likely” and “consistent with” were generally considered to have a variable agreement in conveying diagnostic certainty [3Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8.
[http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] ]. However, although radiologists perceive that there is a variable agreement in conveying certainty in using these phrases, in this study, these were frequently encountered phrases in radiology reports.
These results confirm previous findings that radiologists frequently utilize phrases which convey diagnostic uncertainty. Multiple publications express the need for minimizing uncertainty and ambiguity when reporting findings [12Chapman WW, Fiszman M, Frederick PR, Chapman BE, Haug PJ. Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia. Acad Radiol 2001; 8(1): 57-66.
[http://dx.doi.org/10.1016/S1076-6332(03)80744-4] [PMID: 11201458] , 22Panicek DM, Hricak H. How sure are you, doctor? A standardized lexicon to describe the radiologist’s level of certainty. AJR Am J Roentgenol 2016; 207(1): 2-3.
[http://dx.doi.org/10.2214/AJR.15.15895] [PMID: 27065212] ]; nevertheless, there continues to be the widespread utilization of these phrases. Uncertainty in diagnostic testing is unavoidable because the sensitivity and specificity of tests is rarely 100%. MRI and CT, for instance, are more accurate for diagnosing metastatic tumors in the peritoneum compared to US. MRI is more accurate for detection of lymph node metastases from ovarian cancer than CT [23Tempany CM, Zou KH, Silverman SG, Brown DL, Kurtz AB, McNeil BJ. Staging of advanced ovarian cancer: comparison of imaging modalities--report from the radiological fiagnostic oncology group. Radiology 2000; 215(3): 761-7.
[http://dx.doi.org/10.1148/radiology.215.3.r00jn25761] [PMID: 10831697] ]. Thus, rather than ignoring uncertainty, it is critical to focus on factors that impact the use of diagnostic certainty phrases, and potentially standardize how certainty is conveyed in reports.
Factors that were significantly associated with utilizing diagnostic certainty phrases with a variable agreement for conveying diagnostic certainty included imaging modality (i.e., X-ray), generating reports without trainees, and individual attending physicians. As expected, there is more frequent use of “good agreement” phrases in MRI and CT reports, compared to X-ray, as these tests typically provide more diagnostic detail than X-rays. There may also be the concurrent use of “good” and variable agreement terms in the same report. However, we focused on comparing usage of “good agreement” terms in each report. Reports generated with trainees compared to those generated by attendings alone did not result in more frequent use of phrases with good agreement. In addition, there was variation in use of “good agreement” phrases between attending physicians.
Communication of uncertainty has been referred to as hedging, with unnecessarily ambiguous expressions [24Waite S, Scott JM, Drexler I, et al. Communication errors in radiology - Pitfalls and how to avoid them. Clin Imaging 2018; 51: 266-72.
[http://dx.doi.org/10.1016/j.clinimag.2018.05.025] [PMID: 29906784] ]. Communicating ambiguity, however, has more often resulted in mismanagement and delayed care [16Berlin L. Pitfalls of the vague radiology report. AJR Am J Roentgenol 2000; 174(6): 1511-8.
[http://dx.doi.org/10.2214/ajr.174.6.1741511] [PMID: 10845472] ]. Thus, radiologists have recognized that this practice does not protect from litigation [24Waite S, Scott JM, Drexler I, et al. Communication errors in radiology - Pitfalls and how to avoid them. Clin Imaging 2018; 51: 266-72.
[http://dx.doi.org/10.1016/j.clinimag.2018.05.025] [PMID: 29906784] ]. Rather, it is driven by the uncertainty that clinicians face [25Bruno MA, Petscavage-Thomas J, Abujudeh HH. Communicating uncertainty in the radiology report. AJR Am J Roentgenol 2017; 209(5): 1006-8.
[http://dx.doi.org/10.2214/AJR.17.18271] [PMID: 28705061] ].
It appears that individual attending physicians need education in order to minimize the use of phrases that do not have “good agreement”. However, educating physicians alone may not be an optimal approach for changing behavior without adequately described standards for communicating diagnostic certainty [26Soumerai SB, McLaughlin TJ, Avorn J. Improving drug prescribing in primary care: A critical analysis of the experimental literature. Milbank Q 1989; 67(2): 268-317.
[http://dx.doi.org/10.2307/3350142] [PMID: 2698446] , 27Lesuis N, den Broeder N, Boers N, et al. The effects of an educational meeting and subsequent computer reminders on the ordering of laboratory tests by rheumatologists: an interrupted time series analysis. Clin Exp Rheumatol 2017; 35(3): 379-83.
[PMID: 28339354] ]. As stated previously, uncertainty is unavoidable. Thus, having the means to convey uncertainty in a way that expresses the degree of certainty, like diagnostic test statistics (i.e. 99% sensitivity is greater than 80%), is necessary. Standardizing phrases for conveying certainty might aid in enhancing communication and minimizing ambiguity [22Panicek DM, Hricak H. How sure are you, doctor? A standardized lexicon to describe the radiologist’s level of certainty. AJR Am J Roentgenol 2016; 207(1): 2-3.
[http://dx.doi.org/10.2214/AJR.15.15895] [PMID: 27065212] , 28Schwartz LH, Panicek DM, Berk AR, Li Y, Hricak H. Improving communication of diagnostic radiology findings through structured reporting. Radiology 2011; 260(1): 174-81.
[http://dx.doi.org/10.1148/radiol.11101913] [PMID: 21518775] ]. This is especially critical for natural language processing and information extraction tools so that findings and disease concepts are described with appropriate certainty modifiers.
This study is limited to phrases that were commonly encountered in our institution and have been previously published. These phrases may vary by geographic location, practice, and institution. We only analyzed reports from a single academic medical center, and our findings may not reflect terminology usage that conveys diagnostic certainty in other institutions. We expect that there is greater variability in terminology usage than identified within one institution alone, highlighting the importance of assessing terminology usage. In addition, the use of diagnostic certainty phrases was not assessed in the context of the findings being described. For example, physicians may be more likely to use phrases with “good agreement” for more benign findings as opposed to findings suggesting malignancy. However, the use of uncertain phrases has been previously shown to lead to ambiguous reports [12Chapman WW, Fiszman M, Frederick PR, Chapman BE, Haug PJ. Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia. Acad Radiol 2001; 8(1): 57-66.
[http://dx.doi.org/10.1016/S1076-6332(03)80744-4] [PMID: 11201458] ].
Further studies to enhance communication between providers by providing terminology to express their degree of certainty for findings are needed. In some cases, developing more standard terminology based on diagnostic confidence in specific imaging findings or patient outcomes may be useful. Standardizing phrases that convey certainty, and limiting the use of phrases with variable agreement among physicians may improve communication of radiology results.
In abdominal imaging reports, more than a third of reports contained at least one diagnostic certainty phrase, however, phrases with good agreement between physicians were infrequently used. Standardization of terminology used to convey diagnostic certainly may help create less ambiguous reports and improve quality of care. Furthermore, it can generate more accurate and reliable information processing tools for research and quality initiatives.
Health Insurance Portability and Accountability Act-compliant study was approved by the Institutional Review Board.
No Animals were used in this research. All human research procedures followed were in accordance with the ethical standards of the committee responsible for human experimentation (institutional and national), and with the Helsinki Declaration of 1975, as revised in 2013.
Not applicable.
The authors declare no conflict of interest, financial or otherwise.
The authors would like to thank Laura Peterson for reviewing the manuscript.
[1] | Friedman PJ. Radiologic reporting: Structure. AJR Am J Roentgenol 1983; 140(1): 171-2. [http://dx.doi.org/10.2214/ajr.140.1.171] [PMID: 6600314] |
[2] | Marcovici PA, Taylor GA. Journal club: Structured radiology reports are more complete and more effective than unstructured reports. AJR Am J Roentgenol 2014; 203(6): 1265-71. [http://dx.doi.org/10.2214/AJR.14.12636] [PMID: 25415704] |
[3] | Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer SE. Is terminology used effectively to convey diagnostic certainty in radiology reports? Acad Radiol 2003; 10(6): 685-8. [http://dx.doi.org/10.1016/S1076-6332(03)80089-2] [PMID: 12809424] |
[4] | Augmenting the impact of technology adoption with financial incentive to improve radiology report signature times. Journal of the American College of Radiology: JACR 2010; 7(3): 198-204. |
[5] | Prevedello LM, Ledbetter S, Farkas C, Khorasani R. Implementation of speech recognition in a community-based radiology practice: effect on report turnaround times. J Am Coll Radiol 2014; 11(4): 402-6. [http://dx.doi.org/10.1016/j.jacr.2013.07.008] [PMID: 24161456] |
[6] | Gollub MJ, Panicek DM, Bach AM, Penalver A, Castellino RA. Clinical importance of reinterpretation of body CT scans obtained elsewhere in patients referred for care at a tertiary cancer center. Radiology 1999; 210(1): 109-12. [http://dx.doi.org/10.1148/radiology.210.1.r99ja47109] [PMID: 9885595] |
[7] | Hatzoglou V, Omuro AM, Haque S, et al. Second-opinion interpretations of neuroimaging studies by oncologic neuroradiologists can help reduce errors in cancer care. Cancer 2016; 122(17): 2708-14. [http://dx.doi.org/10.1002/cncr.30083] [PMID: 27219108] |
[8] | Leung JW, Margolin FR, Dee KE, Jacobs RP, Denny SR, Schrumpf JD. Performance parameters for screening and diagnostic mammography in a community practice: are there differences between specialists and general radiologists? AJR Am J Roentgenol 2007; 188(1): 236-41. [http://dx.doi.org/10.2214/AJR.05.1581] [PMID: 17179372] |
[9] | Langlotz CP, Meininger L. Enhancing the expressiveness and usability of structured image reporting systems. 2000; In: Enhancing the expressiveness and usability of structured image reporting systems. Proceedings / AMIA Annual Symposium AMIA Symposium.; 2000; pp. 467-71. |
[10] | Prevedello LM, Farkas C, Ip IK, et al. Large-scale automated assessment of radiologist adherence to the Physician Quality Reporting System for stroke. J Am Coll Radiol 2012; 9(6): 414-20. [http://dx.doi.org/10.1016/j.jacr.2012.01.014] [PMID: 22632668] |
[11] | Krauthammer M, Hripcsak G. knowledge model for the interpretation and visualization of NLP-parsed discharged summaries. 2001; In: Proceedings / AMIA Annual Symposium AMIA Symposium; 2001; pp. 339-43. |
[12] | Chapman WW, Fiszman M, Frederick PR, Chapman BE, Haug PJ. Quantifying the characteristics of unambiguous chest radiography reports in the context of pneumonia. Acad Radiol 2001; 8(1): 57-66. [http://dx.doi.org/10.1016/S1076-6332(03)80744-4] [PMID: 11201458] |
[13] | Gunn AJ, Tuttle MC, Flores EJ, et al. Differing interpretations of report terminology between primary care physicians and radiologists. J Amer College Radiol: JACR 2016; 13(12 Pt A): 1525-9. |
[14] | Hobby JL, Tom BD, Todd C, Bearcroft PW, Dixon AK. Communication of doubt and certainty in radiological reports. Br J Radiol 2000; 73(873): 999-1001. [http://dx.doi.org/10.1259/bjr.73.873.11064655] [PMID: 11064655] |
[15] | Berlin L. Communicating findings of radiologic examinations: Whither goest the radiologist’s duty? AJR Am J Roentgenol 2002; 178(4): 809-15. [http://dx.doi.org/10.2214/ajr.178.4.1780809] [PMID: 11906852] |
[16] | Berlin L. Pitfalls of the vague radiology report. AJR Am J Roentgenol 2000; 174(6): 1511-8. [http://dx.doi.org/10.2214/ajr.174.6.1741511] [PMID: 10845472] |
[17] | van Riel SJ, Ciompi F, Jacobs C, et al. Malignancy risk estimation of screen-detected nodules at baseline CT: Comparison of the PanCan model, Lung-RADS and NCCN guidelines. Eur Radiol 2017; 27(10): 4019-29. [http://dx.doi.org/10.1007/s00330-017-4767-2] [PMID: 28293773] |
[18] | Rosenkrantz AB, Oto A, Turkbey B, Westphalen AC. Prostate imaging reporting and data system (PI-RADS), version 2: A critical look. AJR Am J Roentgenol 2016; 206(6): 1179-83. [http://dx.doi.org/10.2214/AJR.15.15765] [PMID: 26913638] |
[19] | Müller-Schimpfle M, Wersebe A, Xydeas T, et al. Microcalcifications of the breast: How does radiologic classification correlate with histology? Acta Radiol 2005; 46(8): 774-81. [http://dx.doi.org/10.1080/02841850500270274] [PMID: 16392601] |
[20] | Feig SA. Role and evaluation of mammography and other imaging methods for breast cancer detection, diagnosis, and staging. Semin Nucl Med 1999; 29(1): 3-15. [http://dx.doi.org/10.1016/S0001-2998(99)80026-9] [PMID: 9990680] |
[21] | Lacson R, Andriole KP, Prevedello LM, Khorasani R. Information from Searching Content with an Ontology-Utilizing Toolkit (iSCOUT). J Digit Imaging 2012; 25(4): 512-9. [http://dx.doi.org/10.1007/s10278-012-9463-9] [PMID: 22349993] |
[22] | Panicek DM, Hricak H. How sure are you, doctor? A standardized lexicon to describe the radiologist’s level of certainty. AJR Am J Roentgenol 2016; 207(1): 2-3. [http://dx.doi.org/10.2214/AJR.15.15895] [PMID: 27065212] |
[23] | Tempany CM, Zou KH, Silverman SG, Brown DL, Kurtz AB, McNeil BJ. Staging of advanced ovarian cancer: comparison of imaging modalities--report from the radiological fiagnostic oncology group. Radiology 2000; 215(3): 761-7. [http://dx.doi.org/10.1148/radiology.215.3.r00jn25761] [PMID: 10831697] |
[24] | Waite S, Scott JM, Drexler I, et al. Communication errors in radiology - Pitfalls and how to avoid them. Clin Imaging 2018; 51: 266-72. [http://dx.doi.org/10.1016/j.clinimag.2018.05.025] [PMID: 29906784] |
[25] | Bruno MA, Petscavage-Thomas J, Abujudeh HH. Communicating uncertainty in the radiology report. AJR Am J Roentgenol 2017; 209(5): 1006-8. [http://dx.doi.org/10.2214/AJR.17.18271] [PMID: 28705061] |
[26] | Soumerai SB, McLaughlin TJ, Avorn J. Improving drug prescribing in primary care: A critical analysis of the experimental literature. Milbank Q 1989; 67(2): 268-317. [http://dx.doi.org/10.2307/3350142] [PMID: 2698446] |
[27] | Lesuis N, den Broeder N, Boers N, et al. The effects of an educational meeting and subsequent computer reminders on the ordering of laboratory tests by rheumatologists: an interrupted time series analysis. Clin Exp Rheumatol 2017; 35(3): 379-83. [PMID: 28339354] |
[28] | Schwartz LH, Panicek DM, Berk AR, Li Y, Hricak H. Improving communication of diagnostic radiology findings through structured reporting. Radiology 2011; 260(1): 174-81. [http://dx.doi.org/10.1148/radiol.11101913] [PMID: 21518775] |