Original Article - (2018) Volume 19, Issue 1
1Department for General, Visceral, Thoracic and Pediatric Surgery, Clinic of Kempten, Kempten, Germany 2Department of Medical Statistics, University of Heidelberg, Heidelberg, Germany 3Department of General and Visceral Surgery, University Hospital of Ulm, Ulm, Germany
Received November 27th, 2017 - Accepted January 26th, 2018
Context According to the current guidelines, adjuvant chemotherapy is an integral part of the treatment concept for advanced pancreatic cancer. Randomised controlled studies showed that postoperative chemotherapy prolongs survival and reduces the recurrence rate. Objective The aim of our project was to analyse the validity of the most important study on this topic (ESPAC-1) by evaluating the data published in four publications (study design 1993, trial progress report 1997, interim results 2001, final results 2004). Design Our detailed analysis of the validity of the ESPAC-1 trial included the study design, recruitment period, patient selection, randomisation, changes in the protocol, participating institutions, homogeneity of subjects, surgical quality, statistical methods and interpretation of results. Results We identified several shortcomings in the patient selection, participating clinics, randomisation, changes in the protocol, surgical quality, statistics and evaluation of the results. Conclusion Although all analyzed papers concerned the same trial, the detailed evaluation revealed many inconsistencies. We therefore conclude that the validity of the ESPAC-1 trial is not sufficient to make a recommendation for adjuvant chemotherapy for advanced pancreatic cancer.
Chemoradiotherapy; Chemotherapy, Adjuvant; Pancreatic Neoplasms; Radiotherapy
Scientific societies have tried to define guidelines for the management of diseases for more than 30 years. For many oncological diseases, treatment recommendations according to the tumor stage were developed on the basis of the results of randomized trials.
Pancreatic carcinoma, for example, is a disease with a very poor prognosis. 46,420 new diagnoses and 39,590 deaths were reported in the USA in 2014. Up to now, only 10 – 20% of the newly diagnosed patients can be resected with curative intention. Efforts to prolong survival continue with a major focus on improving the effectiveness of the available treatment. According to the current guidelines, adjuvant chemotherapy is an integral part of the treatment concept for patients with resected pancreatic cancer because the 5-year-survival rate rarely exceeds 20% even after curative resection [1].
One of the most important studies for the implementation of adjuvant treatment in international guidelines was the ESPAC-1 trial. This study was designed in 1993 [2] and included patients with resected pancreatic cancer who were candidates for adjuvant therapy.
Results of the trial were published in 1997 [3], 2001 [4] and 2004 [5]. All publications drew the conclusion that adjuvant chemotherapy should be recommended for patients with advanced, but resectable, pancreatic cancer.
It is the aim of this paper to reevaluate the validity of this extensively cited prospective, randomized clinical trial and to analyse whether the results still justify the recommendation of adjuvant chemotherapy in international guidelines.
We used the existing four publications of the ESPAC-1 trial: original study protocol (1993) [2], trial progress report (1997) [3], interim results (2001) [4] and final results (2004) [5]. We compared the reported study design, recruitment period, selection of patients, randomisation, participating centres, changes in the protocol, homogeneity of subjects, surgical quality, statistical methods and evaluation of the results.
Quality-of-life data assessed according to the study protocol were not included into our reevaluation.
Description of the Original ESPAC-1 Protocol Of 1993
The ESPAC-1 trial was designed as a multicenter randomized study using a two-by-two factorial design to investigate whether an adjuvant treatment (chemotherapy for 6 months with 5-FU and D-L folinic acid versus radiotherapy with 40 Gray and 5-FU as radiosensitiser or a combination therapy of both) can improve the 2-year survival of patients with resected pancreatic cancer.
Inclusion criteria were patients with macroscopically resected and histologically confirmed adenocarcinoma of the pancreas, regardless of positive or negative resection margins and the nodal stage.
Exclusion criteria were ascites, liver metastases, tumor spread to distant abdominal organs, peritoneal or omental seedlings or distant metastases, previous or concurrent malignancies (except basal-cell carcinoma of the skin or carcinoma in situ of the cervix) and serious medical or psychological conditions precluding adjuvant therapy.
Randomisation was planned to be performed by the central Pancreatic Cancer Trial Offices in the UK, Germany and Switzerland. Patients were randomized in two steps, first to radiotherapy or observation and second to chemotherapy or observation. Unfortunately, no details are reported on these two randomisation steps.
In the primary protocol, a stratification was mentioned which can be performed according to the presence or absence of tumour infiltration at the resection margins, but it was not described whether patients were stratified before or after randomisation. Furthermore, it was mentioned that “if a participating surgeon felt strongly that his patient should not be subjected to combination therapy, he could choose to randomize between observation and radiotherapy only or observation or chemotherapy only.”
The trial was designed to show an improvement in the 2-year-survival rate from 20 to 40% in patients with negative resection margins and from 1 to 20% in patients with positive resection margins by one or both of the adjuvant treatments (chemotherapy and/or radiotherapy).
According to the power calculation, 280 patients were necessary to answer the study question (Table 1). Given an alpha of 0.05 and a power of 90% to detect such a difference, 220 patients with negative resection margins (55 in each arm) and 60 patients with positive resection margins (15 in each arm) were required in the 2×2 factorial design. This design was planned to compare two pairs of treatment groups in different settings:
A+B versus C+D (no chemotherapy versus chemotherapy) and A+C versus B+D (no radiotherapy versus radiotherapy).
Desription of ESPAC-1 Trial Progress Report 1997
The ESPAC-1 study started in February 1994 and recruited a total of 348 patients from 49 centers: UK (n=85), Italy (n=66), Switzerland (n=62), Germany (n=34), Greece (n=30), Hungary (n=32), Spain (n=19), Sweden (n=7), Austria (n=7) and France (n=6) until May 31, 1997. Out of these 348 patients, 322 patients had a ductal adenocarcinoma of the pancreas, and 26 patients had other types of cancer.
Out of those 322 patients with ductal adenocarcinoma of the pancreas:
−− 175 were randomized to the 2×2 design (any of the 4 arms),
−− 47 to radiotherapy only (versus control) and
−− 126 to chemotherapy only (versus control).
It was stated that 26 patients with periampullary cancer were randomized separately. The distribution to the various treatment arms was illustrated in two tables as shown here in Table 2.
For the N=161 group, it was shown that this is the number of patients assigned to the two-by-two design after exclusion of the patients with periampullary cancer, but the second part of the table is completely confusing because it summarises N=322 at the buttom, but allocates a total of 483 patients to different regiments.
The number of 483 patients can be reached if one adds 161 on the left part of the table to 322 on the right part of the table, but it still remains unexplained why 483 patients are mentioned when only 348 patients were allocated in 1997.
The data presented become even more confusing when one realizes that out of 348 patients included in this randomized trial, only half of them underwent the planned randomization. One explanation might be the influence of the individual doctor’s preference allowing a selected allocation to treatment groups.
This preference could have been institutional (e.g.no radiotherapy available) or individual (e.g. the patient’s general condition).
Furthermore, a different section for patients with periampullary cancer was established separately.The change in the recruitement strategy was followed by a discussion of the calculated number of patients needed to complete the trial. As a result, a number of 480 patients was now calculated.
The paper stated a conflict of interest. Some members of the ESPAC-1 study group were engaged in clinical trials sponsored by British Biotech, which produces marimastat (BB2516), and Lilly Oncology, which produces gemcitabine (Gemzar).
Description of Interim Results 2001
The interim report describes the survival rates for patients recruited between February 1994 and April 2000. The recruitment period was originally planned for two years, but had to be extended. At least 61 cancer centres in 11 countries recruited 541 patients with resected pancreatic cancer. This means that only 1.5 patients were recruited in each participating centre per year. The paper declared that, owing to the large number of European centres involved, a central audit was not performed.
The primary outcome measure was the two-year-survival rate. Secondary outcomes were the incidence of adverse effects and recurrences and quality of life. All analyses were carried out according to the intention-to-treat principle.
As already described in the study progress report, the study design was changed to allow the inclusion of additional patients. In this publication, 541 eligible patients with pancreatic ductal adenocarcinoma were reported to be randomized as follows:
• 285 patients in the two-by-two factorial design,
• 68 patients in radiotherapy versus no radiotherapy and
• 188 patients in chemotherapy versus no chemotherapy.
This strategy led to a variety of subgroups, as shown in Figure 1.
Figure 1. ESPAC-1 trial subgroups according to the interim report 2001.
The term ”radiotherapy” from the original protocoll in 1993 and the first paper in 1997 was changed to ”chemoradiotherapy” in this publication and the following in 2004 without further explanation.
The statistical analyses combined patients of the 2×2 design with additional groups. Analysis of the effect of no radiotherapy vs. radiotherapy showed that the following groups were included in the statistic: 69+74+35 vs. 70+72+33; resulting in groups of 178 vs. 175 patients.
For evaluation of the effect of no chemotherapy versus chemotherapy, several subgroups (69+70+96 vs. 74+72+92; 235 vs. 238) also underwent a combined analysis, so that several subgroups were included twice in different combinations.
The reader gains the impression from this design that the study reports on 826 patients (175+178+238+235=826), but at the time of evaluation, only 541 patients were included, and 314 events were registered. Median follow-up of the 227 patients still alive was 10 months (range 0-62).
Overall results of the combined group analysis showed no survival benefit for adjuvant radiotherapy (median survival 15.5 months in 175 patients with radiotherapy vs. 16.1 months in 178 patients without, p=0.24). Analysis of the patients included into the two-by-two-factorial design (N=285) was also interpreted as showing no significant difference (p=0.09) for the overall survival.
In contrast, adjuvant chemotherapy (median survival 19.7 months in 238 patients with chemotherapy vs. 14.0 months in 235 patients without, p=0.0005) influenced the outcome significantly, as reported from the combined group analysis. Surprisingly, the analysis of the twoby- two factorial design (N=285) could not identify any survival benefit (p=0.19), but the authors described a nonsignificant trend.
This interpretation seems to be highly subjective because the authors consider a p=0.19 to be a nonsignificant trend, whereas a p=0.09 is declaired to be not significant.
The main difficulties with the reported data are:
−− The 541 eligible patients were not allocated to a single, but to three different randomized trials.
−− None of these three trials was analysed separately.
−− Treatment groups derived from three different trials were combined for the statistical analysis.
Authors’ potential conflicts of interest were not mentioned.
Description of Final Results 2004
The final results of the ESPAC 1 trial were published in 2004. The authors focused only on the data of patients randomized to the two-by-two factorial design. The paper reports a total of 289 patients in 53 hospitals in 11 European countries (0,9 patients/centre/year) who underwent randomisation into one of the four treatment groups between February 1994 and June 2000. At the time of final analysis, 237 of 289 patients had died, and a median follow-up of 47 months for 52 surviving patients was recorded.
For the statistical analysis, two of the randomized groups were combined and evaluated twice in different combinations (Figure 2). This kind of evaluation creates the impression that 588 patients instead of 289 were evaluated.
Figure 2. ESPAC-1 study groups at final analysis (2004).
While the study was primarily designed to show an improved survival rate after two years of 20%, the final report now presents also the 5-year-survival data.
The median survival (95% confidence interval) of the four different groups was:
16.9 months for observation (12.3 – 24.8; n=69),
13.9 months for radiotherapy (12.2 – 17.3; n=73),
21.6 months for chemotherapy (13.5 – 27.3; n=75) and
19.9 months for radiotherapy+chemotherapy (14.2 – 22.5; n=72).
The corresponding 5-year-survival rates were 11 %, 7%, 29% und 13%.
Whereas the median survival analysis included the 95% confidence interval, which shows a broad variation in each of the four groups, the 5-year-survival estimates reported only mean values, but neither confidence intervals nor p-values. The reason given for not reporting these data was a lack of statistical power to compare the four groups of the two-by-two trial directly.
The no-radiotherapy group seems to have an advantage if you examine the forrest plot on the hazard ratio for death comparing radiotherapy (radiation alone or in combination with chemotherapy) with no radiotherapy (chemotherapy or observation).
The chemotherapy group seems to have an advantage if you examine the forrest plot on the hazard ratio for death comparing chemotherapy (chemotherapy alone or in combination with radiotherapy) with no chemotherapy (radiotherapy or observation).
Applying this statistical design with double analyses of the four patient groups in different settings, a statistically significant survival benefit was described for patients assigned to chemotherapy, whereas no significant benefit for radiotherapy could be shown. This significant difference in favour of chemotherapy could not be shown in the interim results after two years in 2001. The different numbers at risk (285 in 2001 and 289 in 2004) might have contributed to this difference.
The increase in participating patients is not explained, but it is obvious that the recruitement period had been changed. Whereas the publication in 2001 reported closing the study with recruitment until April 2000, the 2004 paper decribed recruitment until June 2000.
Although the authors declared that a direct comparison of the four groups was not possible due to a lack of sufficient statistical power, our recalculation of results demonstrated the three following significant differences:
Chemotherapy versus Radiotherapy+Chemotherapy: p=0.0124
Chemotherapy versus Radiotherapy: p=0.0004
Chemotherapy versus Observation: p=0.0088
Three other comparisons show no significant results:
Radiotherapy versus Radiotherapy+Chemotherapy: p=0.2494
Radiotherapy versus Observation: p=0.3271
Observation versus Radiotherapy+Chemotherapy: p=0.8689
Dr. Neoptolemos reports having received grant support from Solvay Pharmaceuticals and KS Biomedix.
Pancreatic cancer is still a disease with a very poor prognosis despite the development of radical operation techniques, including extended lymphadenectomy, introduction of perioperative chemotherapy or radiochemotherapy, improved perioperative intensive care and improved diagnostic tools during the last 30 years.
Large registries report an improvement in the 5-yearsurvival rate for all patients from 3% to 8% in the USA or up to 10% in Japan, but the reasons for this improvement remain unclear [1, 6].
It was the aim of the ESPAC-1 study to find out if postoperative chemo-, radio- or chemo-radiotherapy can improve the survival rate in patients with advanced, but still resectable, pancreatic cancer.
The published data of the ESPAC-1study convinced the medical community to recommend postoperative chemotherapy and to introduce this recommendation into several European and international guidelines. The ESPAC- 1-Study is cited for example in the following guidelines: ESMO/ESDO (Europe), S3-AWMF-Guidline (Germany), KCE (Belgium) and NICE (UK) - each with a high grade of recommendation [7, 8, 9, 10].
Considering the still poor survival rates for patients with advanced, but resectable, pancreatic cancer, as well as reflecting on the current discussion about a “choosing wisely strategy” to minimize the patients’ risks, especially for patients with a limited live expectancy, and the financial burden on healthcare systems, we analysed the reported data of the ESPAC-1 study in detail to find out if the study was strong enough to justify the guidelines’ recommendations according to this study.
Our detailed analysis detected the following weaknesses:
−− Large variability of institutions, unclear patient selection: 53 hospitals in 11 countries included patients, no surgical standards defined; long recruitment period, low recruitment rate although high-volume centers participated; no central audit.
−− Changes in protocol: The recruitment period of 2 years was extended to 6 years. The possibility of a preference-based randomization was introduced. The primarily prospective randomized trial was combined with the design of a pragmatic trial. The observation period - planned for two years - was extended to five years because no statistical significance could be observed in the 2 x 2 factorial design group after two years.
−− Randomization: Introducing a preference-based study arm could have contributed an additional sampling bias, which implies that patients with different risk profiles might have been consciously assigned to different therapeutic options.
−− Changes in terminology: The term ”radiotherapy” from the original protocol in 1993 and the first paper in 1997 was changed to ”chemoradiotherapy” in the publications in 2001 and 2004 without further explanation.
−− Questionable study design (2×2 factorial) and statistical analysis: Since each patient is recorded in two groups, the information on each patient is used to answer two questions. This situation is comparable with a multiple-testing procedure, which might lead to an overestimation of observed effects. Another shortcoming of the studies is the failure to test each individual treatment group, which was possible as shown above. P-values are only given for groups in which all patients were included at least twice.
−− Study question not answered: The study was established to answer the question whether an adjuvant therapy is able to increase the survival rate of patients after pancreatic resections by 20% after two years. This 20% increase in survival was presumed for patients with negative, as well as for patients with positive resection margins, but was never analysed in the published papers.
Many patients have undergone adjuvant chemotherapy since the results of the ESPAC-1 trial were published. A prolongation of the survival interval might have been achieved in an unknown number of cases, but side effects with impaired quality of life also occurred. Furthermore the costs for the healthcare systems have to be taken into consideration.
It remains unclear whether the response to chemotherapy and/or the prolonged survival after resection of advanced tumor stages is more related to the tumor biology than to a specific treatment option.
Meanwhile two new studies, the ESPAC-4-Study [11] and the CONKO-001-Study [12], where published on a similar topic as the ESPAC-1-Study. Unfortunately both studies show similar lacks of validity like the ESPAC-1- Study:
The aim of the ESPAC-4-Study was to compare the combined gemcitabine and capecitabine chemotherapy with gemcitabine monotherapy in patients with resected pancreatic cancer. Between 2008 and 2014 372 patients from 98 hospitals were recruited. The result of the study showed a median overall survival of 28,0 months for patients in the gemcitabine and capecitabine group compared with 25,5 months for patients in the gemcitabine monotherapy group (p=0,032). Because each hospital recruited in average only 1,2 patients per year a standardized surgical treatment is unlikely and a high selection bias has to be suspected.
The CONKO-001-Study recruited 368 patients from 88 institutions in Germany and Austria over a period of 6 years. The result showed a significant increase of the disease free survival time for those patients treated with gemcitabine for 6 months after pancreatic cancer resection (13,4 months) compared with an observation group (6,7 months). Again the weakness of the study is the small number of patients enrolled per year (0.7 patients) by each institution which might include a selection bias as well as a lack of probability regarding a standardized operative treatment.
The Japanese Pancreas Tumor Registry [6] analysed the courses of several thousands of patients after pancreatic cancer resections with and without postoperative chemotherapy. The results show a slight improvement in the survival rates over the decades, but they are far behind an improvement of 20% as estimated in the ESPAC-1 trail. It remains completely unclear how such a result might have been achieved.
Data from the American Cancer society [1] also show a minimal improvement of the survival rates over the last decade but many factors, like standardization of the operative procedure, the anesthesia, perioperative management and technical developments might have contributed to the progress despite the introduction of adjuvant therapy.
The aim of our detailed analysis of the ESPAC-1-Study was to take a critical look on the validity of that study which is still mainly cited in many guidelines as a reference for the recommendation of adjuvant treatment in patients after resection of pancreatic cancer.
We therefore conclude that, according to the weakness of the study, as well as more than 10 years of clinical experience in the meantime, the recommendation for adjuvant chemotherapy after resection of pancreatic cancer has to be re-evaluated. Stratification according to defined risk factors, like tumor characteristics, should be introduced to identify possible responders to therapy and thereby reduce the number of unnecessary treatments, particularly because the clinical approach to oncological patients has switched from standardized to personalized medicine.
We focused on the ESPAC-1-Study because in many European and international guidelines this study is still basis of the recommendation of adjuvant chemotherapy after resection of pancreatic cancer. Our detailed analysis of the ESPAC-1 trial revealed several shortcomings and inconsistencies in the study design, patient selection, randomization, statistical methods, recruitment period and interpretation of the results. The conclusion is that a general benefit from adjuvant chemotherapy in resectable pancreatic cancer cannot be derived from this study.
For non of the three authors exists a conflict of interests.
