Original Article - (2017) Volume 18, Issue 5
Department of Anatomical Pathology, Singapore General Hospital, Singapore
Received July 25th, 2017-Accepted September 05th, 2017
Objectives The Papanicolaou Society of Cytopathology’s terminology scheme for endoscopic ultrasound-guided fine needle aspiration cytology includes an ‘atypical’ interpretation category. We conducted a retrospective study to determine the frequency of its use within our institution and the interobserver agreement among pathologists with variable experience in interpretation. Methods Following IRB approval, diagnoses of endoscopic ultrasound-guided fine needle aspiration procedures over a 6-year period, imaging studies and follow-up biopsy/resection results were collated. Eight pathologists blindly reviewed 18 ‘atypical’, 5 ‘benign’ and 5 ‘malignant’ cases using standard reporting terms. The free-marginal kappa (Kfree) was calculated to determine interobserver agreement. The ‘atypical’ reporting rate for each pathologist was also calculated. Results Of 598 pancreatic endoscopic ultrasound-guided fine needle aspiration, 29 (4.89%) were reported as ‘atypical’; In blinded review, Kfree showed slight agreement with the ‘atypical’ category (0.07) and fair agreement with the ‘benign’ and ‘malignant’ categories (0.34 and 0.22, respectively). Combining ‘suspicious’ and ‘malignant’ diagnoses increased the Kfree to 0.50 (moderate agreement). The pathologists had an acceptable mean ‘atypical’ reporting rate of 5.92% (standard deviation: 5, range: 12.2-0.7%). Conclusions The inter-observer agreement level for ‘atypical’ cases was lowest of all groups. Tracking atypical rates may be useful to measure the quality of the endoscopic ultrasound-guided fine needle aspiration interpretation service and as a measure of quality assurance in routine practice.
Pancreas; Endoscopic Ultrasound-Guided Fine Needle Aspiration
The role of pancreatic endoscopic ultrasoundguided fine needle aspiration (EUS-FNA) cytology in diagnosis and management of pancreatic lesions is wellestablished. The Papanicolaou Society of Cytopathology recently proposed a standardized terminology scheme for pancreatobiliary specimens that correlate cytological diagnosis with biological behavior and increasingly conservative patient management of surveillance only [1]. The proposed terminology scheme recommends a sixtiered system which includes an ‘atypical’ interpretation category. ‘Atypical’ diagnoses are rendered in cases where qualitative cytologic features fall short of malignancy and overlap with those seen in benign processes. Low cellularity and premalignant changes (dysplasia) may also be included within this category. Additionally, in many instances, gastrointestinal tract contamination is not easily distinguished from a mucinous neoplasm of the pancreas. The presence of atypical cells in a background of benign pancreatic tissue, where a neoplasm or malignancy cannot be confidently excluded, is also well-placed within this interpretation category [2]. The risk of malignancy of ‘atypical’ diagnoses has been shown to range from 25% to 100% [1, 3, 4, 5]. Alston et al. reported the risk of a pancreatic neoplasm ranges from 6% to 93%, a reflection of the range of benign to malignant/neoplastic lesions represented by this diagnosis [6].
The existing literature reports a mean ‘atypical’ reporting rate for EUS-FNA ranging from 1% to 14% (mean 5.3%) [4]. However, these studies do not comment on the experience of the reporting pathologists in interpreting these challenging cytologic specimens. We conducted a retrospective study to determine the frequency of use of ‘atypical’ in pancreatic EUS-FNA within our institution and to determine the interobserver agreement on a uniform test set of EUS-FNA smear samples in a group of pathologists with variable experience in pancreatic cytopathology smear interpretation.
Following Institutional Review Board approval, cases of EUS-FNA procedures performed from January 2007 to December 2012 were extracted from the pathology database. Other recorded clinical data included: age, sex, initial and follow-up radiological findings (either from CT scans, MRI scans or ERCP findings), EUS-FNA diagnoses and follow-up biopsy/resection information, where available. Fluid chemistry levels (for example, CEA and amylase) were not available at the time of the study.
EUS-FNA procedures were performed under conscious sedation using 22 gauge needles with a cytotechnologist present for on-site evaluation of adequacy. For on-site adequacy, only DiffQuik (Baxter, McGraw Park, IL) staining was used. Final pathology diagnosis was performed with additional Papanicolaou-stained slides as well as hematoxylin and eosin stained cell blocks where available.
Five hundred and ninety-eight EUS-FNA cases were reported during this time period as ‘benign’, ‘malignant’ or ‘atypical’. ‘Benign’ cases included those with normal tissue like pancreatic acinar (grape-like) and ductal (monolayered sheet with well-spaced nuclei), inflammation and reactive processes with no significant cytologic atypia. ‘Malignant’ cases included specimens with unequivocal diagnostic features of malignancy including crowded sheets and clusters of cells with significant cytoarchitectural and nuclear atypia, discohesive single abnormal cells, mitoses and necrosis. ‘Atypical’ cases displayed features as described earlier (Figure 1). From the 598 EUS-FNA cases, twenty-eight were selected for review within this study, including 18 cases with ‘atypical’, 5 cases with ‘benign’ and 5 cases with ‘malignant’ diagnoses. Cases reported as ‘nondiagnostic’ or ‘suspicious’ were excluded from the initial selection. Eight participating pathologists with variable experience in evaluating cytology specimens (3 to 20 years) blindly reviewed these cases using standard terms in reporting categories used within our institution: ‘nondiagnostic’ (unsatisfactory/inadequate), ‘benign’, ‘atypical’, ‘suspicious’ (suspicious for neoplasm) and ‘malignant’ (positive for neoplasm). The free-marginal kappa (Kfree) [7] was calculated to determine the interobserver agreement. The ranges of agreement are as follows: -1.0: perfect agreement, 0.0: agreement equal to chance, 0.01-0.2: slight agreement, 0.21-0.4: fair agreement, 0.41-0.6 moderate agreement, 0.61-0.8: substantial agreement and 0.81-1.0: almost perfect agreement.
Figure 1. (a, b). ‘Atypical’ cells on cytology, and inflammation on final histology. (DiffQuik, magnification 20x; Papanicolaou stain, magnification 10x). (c, d). ‘Atypical’ cells on cytology and adenocarcinoma with mucinous and papillary features on final histology (DiffQuik, magnification 40x; Papanicolaou stain, magnification 20x).
The ‘atypical’ reporting rate for each pathologist over the 6 year time period was also calculated by dividing the number of ‘atypical’ diagnoses made by the individual pathologist over the total number of cases reported for the year.
Five hundred and ninety-eight pancreatic EUS-FNA cases were screened during the study period of 6 years (2007 to 2012), of which 29 (4.9%) were reported as atypical. Twenty-eight cases were selected for blinded review. These patients included 15 males and 13 females with a mean age of 62 years (range: 35-84). Radiological information was available for twenty cases: 16 with a head of pancreas mass, 1 with an ampulla of Vater mass and 3 pancreas body/tail masses by imaging studies. All, except one, were described as solid hypodense masses (Table 1). Of the 28 cases reviewed, 18 were originally diagnosed as ‘atypical’ on cytology. Thirteen (72%) of these cases were malignant on histologic follow-up, 1 was benign and 4 did not have follow-up tissue diagnoses. Following blinded review by 8 pathologists with variable experience in interpreting such smears, the free-marginal kappa (Kfree) was calculated for the ‘overall’ 28 cases, 18 ‘atypical’, 5 ‘benign’ and 5 ‘malignant’ cases. Slight agreement (Kfree 0.15 and 0.07, respectively) was observed with the ‘overall’ and ‘atypical’ categories and fair agreement (Kfree 0.34 and 0.22, respectively) was observed with the ‘benign’ and ‘malignant’ categories (Table 2). As ‘suspicious’ and ‘malignant’ diagnoses were used almost interchangeably among the pathologists, the two interpretations were combined and the Kfree calculated comparing the following diagnostic categories: ‘non-diagnostic’, ‘benign’, ‘atypical’ and ‘suspicious/ malignant’. The Kfree increased to 0.50 (moderate agreement) for a ‘suspicous/malignant’ diagnosis (Table 3).
Five hundred and ninety-eight EUS-FNA cases were reported between January 2007 and December 2012. The yearly atypical reporting rate fluctuated between 1.6% to 7.0% with an average of 5.1% (standard deviation: 1.83) (Table 4). Figure 2 shows the percentage distribution of pancreatic EUS-FNA cases reported by each of the 8 pathologists involved in the current study, as well as other pathologists within the department who did not report any atypical cases during the study period. The 8 pathologists individually reported between 3 (1%) and 140 (24%) EUS-FNA cases within the study period. The mean ‘atypical’ reporting rate was 5.9% (standard deviation: 5, range: 12.2-0.7%); two pathologists who reported less than 20 EUS-FNA cases were excluded (Figure 3).
Figure 2. Percentage distribution of pancreatic EUS-FNA cases reported by each pathologist, ‘A’ to ‘H’ (‘Others’ refer to pathologists that did not report any atypical cases from 2007 to 2012).
Figure 3. Atypical reporting rate of eight pathologists, ‘A’ to ‘H’ (pathologists ‘D’ and ‘E’ were excluded as they reported less than 20 pancreatic EUS-FNA cases from 2007-2012).
In their review of EUS-FNA samples reported within their institution, Saieg et al. applied the newly proposed standardized terminology scheme to 155 cases reported over a 12-month period. Twenty-nine (18.7%) ‘atypical’ diagnoses were re-interpreted as ‘neoplastic-other’ as these cases were of cystic neoplasms with no features of high grade dysplasia [8]. This reiterates the importance of knowledge of radiological findings when interpreting such specimens. Cyst fluid CEA and amylase levels are also helpful to the cytopathologist in rendering an accurate diagnosis [9, 10]. This additional information would no doubt have improved the interobserver agreement in this study. However, the clinico-radiological findings were not made available to the reviewing pathologists in this study in order to assess interpretation differences based solely on the cytomorphologic findings. In the case of cystic lesions, fluid CEA and amylase levels were not recorded as those tests were not available within our institution at the time of the study. Additionally, the interpretation categories were kept at 5 tiers as they are the standard reporting terms the pathologists used in our institution before the introduction of the fairly new 6-tier system. For the purposes of our study, which was to assess differences in the ‘atypical’ interpretation, the exclusion of a ‘Neoplasticbenign/ other’ category was deemed acceptable.
The inter-observer agreement level for benign and malignant control cases was ‘fair’ (0.34 and 0.22, respectively). This was lower than expected and could be attributed to the wide range of interpretive experience (3 to 20 years) and confidence level among the participating pathologists. The ‘atypical’ reporting rates recorded in the literature range from 1% to 14% (mean, 5.3%) [4]. The mean ‘atypical’ reporting rate of 5.9% within our institution for the 6 year study period falls well within this range. The differences in individual ‘atypical’ reporting rates amongst cytopathologists were remarkably different, although the number of cases reported by each pathologist also varied widely (Figure 3), reiterating the fact that likely contributing factors for these differences include frequency of interpreting such cases, the varying years of practice experience, confidence level and expertise in pancreatic cytopathology smear interpretation, excluding the limitations due to the inherent nature of the lesion or material itself [11]. This was similarly shown by Schneider et al. who reported improved rates of sensitivity and negative predictive value of EUS-FNA performed at a specialized cytology institute compared to a standard pathology service [12].
In our study, the overall inter-observer agreement for the 28 pancreas EUS-FNA smears after review was poor, especially for the 18 atypical smears. It improved after combining the ‘suspicious’ and ‘malignant’ categories together, especially for the 5 malignant cases, which suggests that, as expected, the pathologists had stronger threshold levels for a ‘malignant’ diagnosis. The interobserver agreement level for the 18 atypical cases was the lowest compared to the other diagnostic groups reviewed (5 benign and 5 malignant cases). There are many well-documented reasons why cytological smears are interpreted as atypical, including low specimen cellularity, gastro-intestinal tract contaminants, poor cell preservation and under-diagnosis [9]. The variability in both the atypical reporting rates and overall poor interobserver agreement amongst reviewing pathologists has implications for practice as well as education. In low-volume centers, cytopathologists and cytopathology trainees who have limited exposure to pancreatic EUS-FNA’s or a limited range of benign to malignant/neoplastic lesions which are sampled need to be cautious in interpretation until adequate experience is gained, and with the hindsight of histologic correlation. Where available, the addition of cell blocks can be useful to provide additional information in rendering a diagnosis. However, these are ever dependent on adequate sampling. We performed a cell block utility study using paired smears and cell blocks resulting in 63 (83%), 9 (11%) and 2 (3%) of 73 cases confidently diagnosed as malignant, benign and non-diagnostic, respectively, with the addition of cell block material and only 2 (3%) cases remained as ‘atypical’ (data not shown).
This is a single center study that has evaluated the difficult area of the ‘atypical’ diagnostic category in pancreatic EUS-FNA. We have established our rates and compared them against the published literature and studied interobserver variations in the reporting of this category. Additionally, quality criteria and thresholds in non-gynecological cytology are not well-established in such specimens. Our results may provide the basis for developing atypical rates as a measure of quality both as a service as a whole as well as for tracking in routine practice as an additional measure of quality assurance.
The overall poor agreement level amongst participating pathologists indicates that occasional interpretation of pancreatic cytology should be discouraged. In order to be a reliable and responsible interpreter of pancreatic EUSFNA cytology samples, regular and frequent exposure to such specimens is essential. The results also emphasize the limitations of evaluating EUS-FNA samples solely based on cytomorphology. Adjunct imaging, fluid chemistry and clinical information are essential to increase diagnostic accuracy. Proper training guidelines and aspects on workload should be established in the development of a high-quality pancreatic EUS service. In addition, development of quality control based on aggregated and individualized reporting rates for the ‘atypical’ may serve as a simple tool for improving and monitoring of a routine EUS service.
Subjects have given their informed consent. This study protocol has been approved by the institute's committee on human research and thus meets the standards of the Declaration of Helsinki in its revised version of 1975 and its amendments of 1983, 1989, and 1996 [JAMA 1997;277:925-926].
The authors have declared that no competing interests exist.
