International Exchange - (2004) Volume 12, Issue 1
General Practitioner and Medical Co-ordinator
Trudy van der Weijden MD PhD
Associate Professor
Willy I Dubois(À) MSc
esearch Assistant Centre for Quality of Care Research, Care and Public Health Research Institute (CAPHRI), Department of Primary Care, Maastricht University, The Netherlands
Ivo JM Smeele PhD
General Practitioner, Medical Co-ordinator, Centre for Diagnostics and Consultation, Elkerliek Hospital, Helmond, The Netherlands
Marianne A Meulepas MSc
Senior Researcher, District Association of Dutch GPs, Eindhoven, The Netherlands
Richard PTM Grol PhD
Professor of General Practice, Centre for Quality of Care Research, Care and Public Health Research Institute (CAPHRI), Department of Primary Care, Maastricht University, The Netherlands
Accepted date: 9 October 2003
Objective Evaluation of the feasibility of an innovative strategy to improve general practitioners’(GPs’) test ordering behaviour, and to further improve continuous professional development.Design Prospective process evaluation of the use and appraisal of the strategy during the first and second years of a trial.Setting General practice, local GP groups, diagnostic centres.Intervention The new strategy combines written feedback, education on clinical guidelines and continuous quality improvement sessions, quality circles,in small local GP groups. An important feature of the written feedback was a comparison of the behaviour of individual GPs with that of their colleagues. Mutual feedback was given by working in pairs; discussion on national guidelines and making plans for change were important features of the group sessions. The strategy has an iterative character.Results All 194 participating GPs received the planned six feedback reports. Data from 156 meetings of 26 local GP groups showed a participation rate of 81% (95% conŽ dence interval (CI): 77–85%) in the first year and 73% (95% CI: 68–77%) in the second year.Meetings included mutual feedback by working inpairs (used in73%of the sessions in the first year and 61% in the second year), individual plans for change (96% in the first year, 92% in the second year) and group plans for change (71%in the first year, 54% in the second year). In the first year GPs expressed their level of satisfaction with the approach with a score of 7.55 on a scale of 0–10(95% CI:7.46–7.64); average score in the second year was 7.51 (95% CI: 7.30–7.74).Conclusion The innovative test ordering strategy seems a feasible tool for continuous improvement of GPs’ test ordering behaviour, Ž fitting in well with local and regional quality improvement efforts for those GPs working in isolated settings.
evaluation studies, feedback, guidelines, healthcare, primary healthcare, professional practice, quality assurance, test ordering
Numbers of tests ordered by general practitioners (GPs) is growing, and inter-doctor variation has been shown to be large.[1–3] It is as yet unclear, however, what would be the best method to influence GPs’ test ordering behaviour.[4] Studies evaluating di¡erent types of interventions and strategies for this purpose have, so far, produced heterogeneous results.[1–5] No particular type of intervention was found to be in-herently e¡ective; multifaceted approaches have proved to be superior to single methods in some analyses, but not in others.[6,7] Audit and feedback were found to be e¡ective in specific settings, while written, personal feedback on test ordering by peers or opinion leaders has also been found to improve test ordering behaviour.[8,9] It seems particularly important in this respect to make use of interventions in addition to professionally oriented interventions, because the success rates of particular strategies seem to be highly dependent on the extent to which they fit in with the local context and the practitioners’ daily work rou-tine.[10] A multifaceted strategy combining comparative feedback on tests ordered, group education on guide-lines, and small group quality improvement meetings in a local GP group, with social influence as an important motivator for change, was expected to o¡er good prospects.[11,12] The strategy also fits in well with the work setting of many GPs in European and non-European countries, which are often characterised by small practices, relatively isolated settings and a desire for more contacts with peers.
The favourable clinical e¡ects of this strategy have been reported elsewhere.[13] Nowadays process evalua-tions of quality improvement strategies are seen as a necessary addition to studies aiming to learn about important elements of change.[14] It was therefore important to determine to what extent the intended elements of the multifaceted strategy were accepted and actually used by the participants and to assess their opinion on the key elements of the feedback and interactive quality circles between colleagues.[15,16] The present paper focuses on the feasibility of this inno-vative strategy in view of a possible implementation on a larger scale, and it also assesses important elements from the perspective of further improving continuous professional development (CPD) of GPs.
Design and subjects
BetweenJanuary1999 andOctober 2000, the newstrat-egy was evaluated in five regions in the Netherlands, and a process evaluation was done prospectively. Co-ordination of the feedback and supervision of the group meetings was provided by the five diagnos-tic centres, which are a special facility where GPs can order laboratory, imaging and function tests without referring patients for specialist care. One of the tasks of the medical co-ordinator of these centres is to give feedback to GPs on their test ordering behaviour. Local GP groups that referred their patients to one of the five participating diagnostic centres were invited to take part in the study. Local GP groups are an existing part of the infrastructure of Dutch GPs collaborating in a specific region. One of their tasks is to organise care during out of o¤ce hours, while CPD is another important activity in many of these local groups.
Intervention: the improvement strategy
The intervention consisted of the following elements: personalised graphical feedback, including a compar-ison of each GP’s own test ordering data with those of colleagues; guideline dissemination and continuous quality improvement meetings in small groups, organised and chaired by the medical co-ordinator of the diagnostic centre. The strategy was patient care-oriented rather than test-oriented, in that it did not focus on the volume of specific tests, but on specific clinical problems and associated laboratory, imaging and function tests relevant to daily GP practice (see Table 1). GPs received three di¡erent feedback reports per year on three di¡erent clinical problems, together with the national, evidence-based guidelines on test ordering of these specific clinical subjects. This was followed by a 90-minute structured meeting two weeks later, at which one of the clinical problems was discussed. The small group meetings or quality circles consisted of three major components. The first was mutual personal feedback by peers, who worked in pairs at the start of the meeting. This was assumed to be a safe method of peer review. The second component was an interactive group education of national guidelines, to enable participants to relate their own and each other’s test ordering behaviour with them. The third was the development of indi-vidual and group plans for change, to stimulate GPs to really put their plans into daily practice. This schedule was repeated a year later, using the same three clinical problems, to assess whether a GP or GP group had implemented the plans for change and to initiate further improvements. This iterative aspect was another important feature of the strategy.
Table 1: Clinical problems and associated tests used in the trial
Variables and instruments
The feasibility of the strategy was tested by a prospec-tive process evaluation, focusing on six variables:
1 the timely production and provision of the feed-back reports
2 the GPs’ appreciation of the feedback
3 the attendance at the meetings
4 the GPs’ appreciation of the meetings.
These four variables were measured by means of a one-page standardised questionnaire, which was com-pleted by the attending GPs after each meeting. Appreciation was measured on a scale of 0–10.
5 With a checklist, the medical co-ordinators recorded actual activities at the meetings, e.g. mutual feedback, discussions on guidelines and plans for change.
6 Individual and group plans for change were drawn up by the participating GPs, written down and collected by the co-ordinators of two regions during the meetings.
Statistical analysis
Analyses were performed separately for the first and second year, in view of the iterative aspect of the intervention. For the same reason di¡erences in attendance between the first and second years were tested for significance using the McNemar test for paired variables. Subgroup analyses for regions and for clinical problems were performed for some of the parameters to see if region and clinical problems were important determinants for the process evaluation. Because there were di¡erences in group size, Spear-man’s correlation coe¤cients were calculated to see if group size was correlated with items from the actual activities questionnaire. Analysis of variance (ANOVA) and multivariate regression analyses were done on the GPs’ appreciation of the feedback reports, using the clinical problem, the region and the local GP group as independent variables.
A total of 37 local GP groups were invited to take part in the trial. The total study population was 193 GPs, belonging to 26 local GP groups that were willing to participate. Individual GP and GP practice character-istics were largely similar to those of the Dutch GP population as a whole, except for type of practice: two-person practices were under-represented, while group practices were over-represented. The mean group size was 7.4 ± 2.7 (SD), range 3–12. A total of 1158 (6 £ 193) written feedback reports were sent out, and 156 small group quality improvement meetings were held. A total of 850 GP questionnaires were analysed, 455 in the first year and 395 in the second year. The response of the participating GPs to the question-naires was 97% in the first year and 93% in the second year. The response of the medical co-ordinators was 100% in the first year and 99% in the second year.
Each participant received all six feedback reports as planned. It proved to be possible to produce and disseminate the feedback in time. The GPs gave a favourable assessment of the feedback reports in both years (see Table 2). Multivariate regression analysis showed that the region where the GP practised, the local GP group and the clinical problem had no significant influence on the appreciation in the first year. In the second intervention year, the clinical problem did influence the appreciation of the report (P = 0.03), in that the appreciation of the feedback report on chronic obstructive pulmonary disease (COPD)/asthma-related tests decreased in the second year. Attendance at the meetings in the first year was on average 81% (95% confidence interval (CI): 0.77– 0.85); in the second year attendance decreased to 73% (95% CI: 0.68–0.77) (P < 0.05, McNemar test). Only two of the 196 GPs never visited any of the meetings. Subgroup analysis showed that there were no significant di¡erences in attendance according to region or clinical problem (P > 0.05). Overall, parti-cipants expressed favourable opinions on the new strategy: the average appreciation score was 7.55 (95% CI: 7.46–7.64; scale 0–10; range 4–10) in the first year and 7.51 (95% CI: 7.38–7.65) in the second.
Table 2: Appreciation of written feedback reports, ® lled in by 193 GPs
Table 3 describes the actual activities in the meet-ings during the two trial years. Discussion of partici-pants’ own test ordering behaviour was performed according to plan in all meetings. As planned, all groups discussed the relationship with the evidence-based guidelines as well as – in the second year – the plans for change made in the first year. In the first year, participants worked more in pairs than in the second year; in two out of the five regions less pair work was undertaken. There was a significant positive correla-tion of 0.38 (P < 0.01) between a larger group size and more pair work in the first year, which disappeared in the second year. Table 3 also shows that GPs made individual plans for change in most meetings. Most groups also made group plans for change, although this decreased in the second year.
Table 3: Activities performed during the meetings in the ® rst and second years, described by medical co-ordinators (according to clinical problem, in percentages)
Table 4 gives the most-mentioned individual plans for change for each clinical problem. Most plans concerned a decrease in the number of tests, except for lung function tests. An example of such an individual commitment was, ‘I will order fewer haemoglobin tests, because I realise that this test does not give much information in patients with vague complaints’. In the second year the number of individual plans decreased, except for the clinical problem general malaise/vague complaints. Plans at group level were also made, e.g. the plan to use the same patient brochure to inform patients about the use of cholesterol tests, or the arrangement to follow the national guideline on delaying testing in patients with vague complaints. All results show that the quality circles were an essential element in the improvement strategy.
Table 4: Individual plans for change made by GPs in two regions during the two-year period
The innovative, multifaceted strategy for improving test ordering behaviour was favourably evaluated by a large GP population. All local GP groups expressed a desire for continuation of the meetings after the experiment. The new strategy utilised peer influence among GPs, and gave GPs the opportunity to openly discuss their test ordering behaviour with colleagues.
The results may be biased, since the study popula-tion di¡ered slightly from the Dutch GP population , but but only in relation to the type of practice. However, there is no reason to assume that these minor di¡erences influenced the external validity of the study.
The decision to focus on clinical problems instead of tests was a good choice, since it allowed the feedback and group work to be linked to national evidence-based guidelines. GPs appreciated this approach, because it was also closely related to their everyday work routine. They stated that this type of feedback definitely had added value, because comparison with colleagues made them more conscious of their own behaviour and motivated them to change.[17,18] Their main criticism was the validity of the numbers of tests in the feedback and the absence of patient-related data. Working in pairs to discuss the feedback report at the start of the meetings made most GPs feel safe, especially in the first year. After a while, it may become less needed, because participants may then feel more safe about discussing their own behaviour within the group as a whole. This is probably why the use of pair work decreased in the second year. Drawing up concrete individual and, if possible, group plans for change that are checked later is a crucial and innovative aspect of this strategy. Most GPs made individual plans for decreasing the numbers of certain tests. However, lack of experience in drawing up and evaluating plans for change made some GPs hesitant about designing such plans. In general, GPs were excited to find in the second year that they had indeed changed in accordance with their plans, and they were then usually more motivated to implement further changes. Nevertheless, individual plans for change were not always adhered to. Making group plans for change can be di¤cult, due to lack of confidence or lack of familiarity with entering into this kind of commitment in a GP group. However, almost two-thirds of the meetings managed to draw up group plans for change. An explanation for the slight decrease in the attendance rate in the second year might be that the same clinical problems were dis-cussed, with some GPs stating that they did not expect to learn anything new, and they preferred to discuss a new clinical subject at each meeting in addition to evaluating previous plans for change.
There is some empirical evidence that participating in quality circles may increase GPs’ job satisfaction, and this powerful, interactive group strategy fits well within the growing need of transparent healthcare with positive use of actual clinical data for CPD in order to further improve clinical practice.[15,19] The following lessons for the CPD of GPs can be learnt. First, GPs appreciate the combination of individual feedback, discussions about guidelines and small group quality improvement meetings driven by peer influence. A second important element is the fact that GPs are prepared to discuss personal, transparent data openly in a group of colleagues. Thirdly, another mportant element is the focus on daily, clinical GP problems. In our study GPs preferred to talk about clinical problems and tests linked to these problems, rather than to discuss abstract phenomena like total test ordering volume or the ordering of specific tests. Finally, the strategy must fit in with the GPs’ daily practice routine and should be aimed at local col-laboration in teams or groups.
None.
The authors gratefully acknowledge the financial con-tribution to the study by the Dutch College for Health Insurances.
