Research Paper - (2005) Volume 13, Issue 4
Clinical Audit Data Manager
Stephen Ashmore BA(Hons)
Manager
Sharon Mandair
GPAQ Facilitator
Leicestershire Primary Care Audit Group, Leicester, UK
Received date: 6 September 2005; Accepted date: 29 September 2005
Objective To facilitate the General Practice Assessment Questionnaire (GPAQ) with local general practice surgeries, with a view to helping achieve Quality and Outcomes Framework points, improving patient experience in primary care, and increasingpatient involvement in health care. Design GPAQ is one of two nationally accredited general practice patient satisfaction surveys and is developed by the National Primary Care Research and Development Centre (NPCRDC) at the Universityof Manchester.Setting The Leicestershire Primary Care Audit Group (PCAG) facilitated this survey for general practices in Leicestershire, Rutland and Northamptonshire, which contains a very varied demographic mix from deprived inner-city areas to very affluentrural communities.Subjects Data were collected for 696 doctors from 183 general practice surveys. A minimum of 50 questionnaires per practitioner were required for statistical validity, so a total of 37 981 questionnaires were analysed.Results Results take the form of scale scores calculated by combining responses to questions on the form. The six scales are ‘Receptionists’, ‘Access’, ‘Communication’, ‘Enablement’, ‘Continuity’ and ‘Overall satisfaction with the service’. Once calculated,these scale scores could be compared with national benchmarks supplied by the NPCRDC. Comparison could also be made between the practice and the PCT, or between PCTs, or with aggregated PCAG benchmarks for all 37 981 samples.Conclusions The GPAQ survey has generated vast amounts of invaluable data concerning patient experiences of primary care, and has proven to be an excellent way of involving patients in healthcare provision.
General Practice Assessment Question-naire, Leicestershire, Primary Care Audit Group
How the GPAQ survey was developed
Some aspects of quality are best assessed by asking patients. The University of Manchester reviewed the available literature to identify aspects of general prac-titioner (GP) care that are most highly valued by patients. These include:
• availability and accessibility, including availability of appointments, waiting times, physical access and telephone access
• technical competence, including the doctor’s know-ledge and skills, and the effectiveness of his or her treatments
• communication skills, including providing time, exploring patients’ needs, listening, explaining, giving information and sharing decisions
• interpersonal attributes, including humaneness, caring, supporting and trust
• organisation of care, including continuity of care, and the range of services available.
In order to assess these aspects of care the Leicester-shire Primary Care Audit Group (PCAG) started from what is regarded as the best available questionnaire, the Primary Care Assessment Survey (PCAS), which had been extensively validated in the United States.[1–4] In collaboration with the Health Institute in Boston, PCAS was modified for use in British general practice. The modified questionnaire was called the General Practice Assessment Survey (GPAS). GPAS has been used in large studies in the UK, and detailed research data on GPAS have been published.[5–9]
For the new GP contract, the National Primary Care Research and Development Centre (NPCRDC) were asked to modify their original GPAS questionnaire, and the General Practice Assessment Questionnaire (GPAQ) was produced. The main difference is that the new questionnaire is shorter. Two versions have also been created, one designed to be sent by post, and one designed to be given to patients after consultations in the surgery. The discussion that follows refers exclus-ively to the consultation version of GPAQ.
PCAG facilitated GPAS for several local primary care trusts (PCTs) in the year 2003–2004, and with the intro-duction of GPAQ in 2004 decided to greatly expand this project to cover all willing practices in Leicester-shire and Rutland, and a large number of the PCTs covering Northamptonshire. In all, 183 practices partici-pated in the project: a total of 696 GPs returned the minimum of 50 questionnaires required, so that a total of 37 981 completed forms were available for analysis.
PCAG printed, distributed and collected the ques-tionnaires. Once returned, the questionnaires were processed using the document scanning system, Formic, and then analysed in a series of stages using custom-designed visual basic for applications tools. The pro-cess was almost completely automated to allow very large volumes of data to be accommodated. Reports were issued to the participating practice on CD along with a suggested action plan for implementing change, and when comparative data became available this was published on the PCAG website at www.leicester- pcag.org.uk. Practices could also use this internet facility to monitor the progress of their data through the system.
Figure 1 shows the difference between the amal-gamated results of all patient responses across the six scale scores. As can be seen, on the whole the region compares well with national averages, and in particu-lar appears to be performing very well in the ‘Recep-tionist’ scale (4.7% higher than the benchmark), ‘Communication’ (5.9% higher) and ‘Overall satis-faction with the service’ (2.5% higher than the corre-sponding benchmark).
Figure 1: Differences between aggregated results and NPCRDC national benchmarks
Results differed widely between local PCT areas too, as one would expect considering the demographic differences present. The ‘Access’ scale score contains results from the questions on the questionnaire relat-ing to patient opinion of waiting times (both for appointments, and for waiting room delays), practice opening hours, how easy it is to get through on the telephone and how patients rate the ease with which they can speak to a GP on the telephone. Figure 2 shows how results for the ‘Access’ scale score vary across the eight participating PCTs. In this graph, and those that follow, DAV indicates Daventry PCT, HB Hinckley and Bosworth PCT, MRH is Melton, Rutland and Harborough, CNWL is Charnwoodand North West Leicester PCT, SL is South Leicester-shire, NH Northampton Heartlands PCT, and LCW and EL are the accepted abbreviations for the two city PCTs, Leicester City West and Eastern Leicester re-spectively. Also shown in Figure 2 and in subsequent graphs are the national benchmarks, and PCAG benchmarks based on aggregated data from 37 981 samples.
Figure 2: Comparison of ‘Access’ scores across PCTs
Results for the ‘Receptionist’ scale score are based on a single GPAQ question, asking patients to grade the quality of the reception staff from ‘very poor’ to ‘excellent’. Again, there is significant variation across results for local PCTs, as shown in Figure 3.
Figure 3: Comparison of ‘Receptionist’ scores across PCTs
It is interesting to note for this case that all practices scored well above the national benchmarks. Note also that rural PCTs seem to score consistently higher than the city-based PCTs of Eastern Leicester and Leicester City West.
‘Continuity’ is the scale score associated with the question on the questionnaire requesting information about how often the patient sees his or her usual doctor and how they rate this. (Details about how often the patient sees any doctor at the practice are covered under the ‘Access’ scale score, above.) The highest-scoring practice in this section scored 92.4%, the lowest 45.2%. The differences in performance of the eight PCTs and a comparison with both the national and PCAG benchmarks are shown in Figure 4.
Figure 4: Comparison of ‘Continuity’ scale scores across PCTs
‘Continuity’, controversially, is one of only two categories of analysis in which the area as a whole performs less well than the standards suggested by the national benchmarks. When we come to discuss patient comments later in this article, we shall see that continuity of care is of great interest to patients, and under the new scheme of advanced access for appointments it is quite often not possible both to meet these targets and to ensure that patients receive continuity of care.
The ‘Communication’ GPAQ scale is calculated from the eight subsections asking for patient opinion on the practitioner’s thoroughness, listening skills, ability to put patients at ease, explanation skills, patience, caring and concern. As Figure 5 shows, all PCTs achieved impressive ‘Communication’ scores. In fact all averaged out to score well above the national benchmarks.
Figure 5: Comparison of ‘Communication’ scale scores across PCTs
The ‘Enablement’ scale score refers to how effective the GP is at involving the patient in his or her own treatment and how well the patient feels his or her doctor has equipped them to understand their prob-lems, and enabled them to keep themselves healthy. These three questions are amongst the least answered (i.e. the ones with the highest number left completely blank). In fact 10%, 13% and 15% of respondents respectively failed to respond to the three questions comprising question 11 of the questionnaire, and it is these three questions that provide data for the ‘Enablement’ scale score. A similar percentage of respondents ticked the ‘Does not apply’ option, compared to 7% for question 10a and only 3% for question 10g. These figures are perhaps also reflected in the fact that, on average, GPs in Leicestershire and Northamptonshire scored lower in this scale than the national benchmark. Apart from ‘Continuity’, this is the only scale score where this is the case. Figure 6 shows a comparison for the results of this scale score for local PCTs, and a comparison with both the national and PCAG benchmarks.
Figure 6: Comparison of ‘Enablement’ scores across PCTs
The last scale score – ‘Overall satisfaction with the practice’ – is also perhaps the most controversial in that many practice staff carrying out the survey noted that patients appeared to be answering the question incorrectly. The NPCRDC has recognised that the layout of question 12 on the survey – ‘All things considered, how satisfied are you with the practice?’
– is incorrect. The layout of preceding questions is such that positive answers lie on the right-hand side of the page, whereas question 12 offers the respondent a sudden and potentially confusing reverse of this trend, with the result that patients who have responded positively to each preceding question in many cases answer ‘Completely dissatisfied’ rather than ‘Com-pletely satisfied’. As part of the analysis of the GPAQ dataset, PCAG examined cases where patients had answered ‘Completely dissatisfied’ to this question, looked at responses to previous questions and judged whether or not the patient had answered incorrectly. Out of these 1030 cases, it was judged that a sur-prisingly large 824 of these had answered the question erroneously, in 186 instances there was an ambiguity in responses that made it impossible to tell, and the remaining 20 patients had clearly intended to tick ‘Completely dissatisfied’. The average ‘Overall satis-faction’ scale score was 80.5% before this critique, but altering the responses as described above changed this by 2.5% to 83%, both of which scores are well above the national benchmark of 78%.
All local PCTs’ averages in this scale score were either equal to or greater than the national benchmark for ‘Overall satisfaction’. The highest-scoring practice in the area scored an impressive 95.3%, and the lowest-scoring 64.8%. A comparison of average re-sults for all local PCTs is shown in Figure 7.
Figure 7: Comparison of ‘Overall satisfaction’ scores across PCTs
Waiting times
Waiting times are obviously a chief patient concern. By far the majority of patients (73%) report that they have to wait between six and twenty minutes in the waiting room, and only 1 in 20 of these decided that this was ‘Very poor’ or ‘Poor’, with 95% claiming they thought this level service ‘Fair’ or better. Indeed, over the entire range of responses, only slightly fewer than 9% of patients thought that waiting times were ‘Poor’ or ‘Very poor’, so it is clear that on the whole patients are either happy about the length of time they wait, or realise the pressure practice staff are under and accept the situation. The vast majority of patients thought that waiting less than five minutes was ‘Very good’ or ‘Excellent’, though waiting times of between six and ten minutes were generally considered to be ‘Good’. Even most of those patients waiting between 11 and 20 minutes considered this treatment ‘Fair’, and sig-nificant numbers of ‘Poor’ or ‘Very poor’ responses only really occur when the length of the wait is more than half an hour.
Questions 4a and 4b on the questionnaire monitor how quickly patients are able to see their usual doctor, and their rating of how happy they are with this service. Similarly, questions 5a and 5b ask for patient opinions about how quickly they are able to see any doctor at the practice, and how they rate this service. Results for these two questions are presented in graphical form in Figure 8. Eighty-three per cent of patients who answered question 4a said that they were able to see any GP within two working days, but only 63% said that this was the case when it came to seeing their usual doctor. This means that 17% of patients are unable to access a GP within two working days, and 36% of patients are unable to access their own, usual GP within the same time period.
Figure 8: Waiting times to see any GP and the patient’s usual GP
As far as satisfaction with this service is concerned, 59% of patients said that access to any GP was either ‘Good’, ‘Very good’ or ‘Excellent’, and 80% of patients rated this non-specific GP access to be better than ‘Fair’. In the case of access to their usual GP, 73% rated their treatment as ‘Good’, ‘Very good’ or ‘Excellent’, and only 13% of patients rated this specific GP access as ‘Poor’ or ‘Very poor’.
In all, 598 patients who responded to question 5a concerning access to any GP at their practice reported that they had to wait more than five days, which amounts to 2.5% of patients.
Further analyses
More elderly patients show a tendency to be happier with primary care services than younger ones, and accordingly answer the questions on the GPAQ survey more positively. Figure 9 shows the results for the ‘Overall satisfaction’ scale score analysed by 10-year age bands, and illustrates this observation quite clearly.
Figure 9: The effect of average patient age on ‘Overall satisfaction’
Another contributory factor for predicting ‘Overall satisfaction’ scores is the number of partners at the practice. On average, single-partner practices scored 81.8% in ‘Overall satisfaction’, which is 1.6% higher than the average for practices with more than one partner at 80.2%. The highest-scoring practice in this scale score (95.3%) was from a GP at a single partner practice. Figure 10 shows the scores for ‘Overall satisfaction’ according to the number of partners at the practice, which shows that single-handers exhibit significantly higher scores than multipartner services. As one might expect, scores for ‘Continuity’ are generally higher for single-partner surgeries as well. In fact single-partner practices scored 15.6% higher than multiple-partner practices – on average, 82.5% compared to 66.8%. Eight out of the ten top scoring doctors for the ‘Continuity’ scale score were single partners. The effect of number of partners on ‘Conti-nuity’ scores is illustrated in Figure 11.
Figure 10: The effect of number of partners in the practice on ‘Overall satisfaction’
Figure 11: The effect of number of partners in the practice on ‘Continuity’
Single-partner practices form exactly one-quarter of all those in the Leicestershire, Rutland and Northamptonshire region, as shown in Figure 12. Practices with one, two or three partners together account for nearly half of the total.
Figure 12: Numbers of GPs in practice across the area
GPAQ Stage two
The intention of all satisfaction surveys is the im-provement of patient satisfaction, and the importance of practices acting on their results was emphasised by PCAG supplying them with template action plans with the feedback reports. Practices were then encouraged to send copies of completed action plans back to PCAG with the dual intention of compiling a library of best practice, and providing further data about how running the GPAQ project has affected service pro-vision.
Such qualitative data is difficult to analyse, but certain popular action points immediately became clear. Thirteen per cent of the 44 practices that returned action plans or meeting minutes stated an intention to improve services by changing the length of GP appointments, and 18% decided to introduce a number of pre-bookable appointments in order to
increase access. Twenty per cent decided they would address ‘Receptionist’, ‘Communication’ or ‘Enable-ment’ issues by provision of training for staff. Provi-sion of more information to patients in the form of leaflets or posters proved the most popular action point, with nearly half (47%) of practices intending to improve service in this area. Twenty-nine per cent decided to make changes to the quality of either the waiting room or practice buildings, and 18% either invested in new telephone systems, or expanded the number of lines available for patients. Finally, three practices (7%) decided to form patient groups as a result of running the GPAQ survey.
These action plans varied greatly in quality, with a strong variety in the number of action points they contained. Four actually contained no action points at all, while one proposed five changes to working practice. Most (73%) proposed one or two action points in the areas described above.
To help practices further with feedback to patients, PCAG also published an eight page newsletter devoted entirely to the GPAQ survey. This document presented aggregated data for the entire area, comparisons be-tween PCTs, a discussion of problems encountered around the ‘Overall satisfaction’ and ‘Enablement’ questions, a brief guide to the technical aspects of the data processing, and a series of small articles contributed by practices who had scored particularly highly in each scale score area. The newsletter also contained tips for successful facilitation of the survey should the practice opt to carry out similar patient satisfaction surveys in subsequent years.
The success of the GPAQ project and the immense help it has made to general practices in Leicestershire and Northamptonshire means that PCAG are facilitating the survey again in 2005, in order to give surgeries an idea of the benefits their action planning and change implementation have caused. PCAG has expanded the area of coverage of data collection to include 784 GPs from 191 surgeries in the area, and is also running the very similar GPAQ for Practice Nurses survey in parallel, which will provide data on 236 nurses from 87 practices across the district. The feedback reports to GPs and practices will thus contain comparative data between this year and last year’s data, enabling staff to evaluate the efficacy of their action planning.
Footnote
As a follow-up to the GPAQ survey, PCAG sent out an auxiliary questionnaire designed to assess practice staff’s opinions on the service provided, asking ques-tions about quality of the final report, timing of delivery and collections, the guidance pack provided and how the practice had fed back the results to patients. This survey covered 115 practices across all but one of the participating PCTs. In all, the service was very well received, with 88% of respondents regarding it as either ‘Good’ or ‘Very good’, 86% considered the guidance pack as ‘Good’ or better, and 89% reckoned the final report to be of a similar standard.
Practices indicated that they had fed the results of the survey back to patients by means of practice newsletters, the practice notice board or website, or by means of an audible health channel in the waiting room. Eighty-seven per cent of practices indicated they would appreciate support in producing a short practice feedback poster for display in their waiting room, and this will be part of the service received in 2005.
Finally, 87% of practices deemed PCAG’s service as ‘Good’ or ‘Very good’ in terms of value for money. In all, it is estimated that PCAG’s facilitation of GPAQ locally has resulted in savings of £20 000–£30 000 to the local healthcare community, compared with the case if practices had decided to carry out the survey through the agency of private companies offering comparable or inferior services.
None.
