Quality in Primary Care Open Access

Keywords

audit, faculty development, family medicine, quality assurance, mammography rates 

Introduction

Quality assurance (QA), the monitoring of practice behaviours to ensure most effective and efficient delivery of healthcare, is an important part of the practice of medicine. The College of Family Physicians of Canada requires that quality assurance be part of the curriculum for all residency programmes in Canada. All practising practitioners are encouraged to review and improve the care they provide, but there are no formal criteria for how this ought to be accomplished and few practice audits are performed in the community.

Lemelin, in a multifaceted continuous quality im-provement (CQI) cluster randomised controlled trial in health service organisations (HSO), had chart abstractors perform baseline and post-intervention assessments. Nurse practitioner facilitators conducted 33 visits to each practice, on average lasting 1 h 45 min each. Modest improvements in preventive care ser-vices in the intervention practices were reported.[1]

Translating this comprehensive CQI intervention into a process that other practitioners could partici-pate in is costly in terms of data gathering, including facilitator and physician time.

The substantial workload of identifying eligible charts and abstracting relevant information deters many practitioners from starting the QA process. A less onerous task might encourage physicians to be involved in QA and CQI activities. While electronic medical records (EMRs) offer the potential to facili-tate data gathering, electronic patient databases are still fairly uncommon in most community-based practices.

Although Lemelin demonstrated that audit with feedback, educational interventions and reminders has some impact on changing physicians’ behaviour, getting physicians to participate in the audit process has been difficult.

The Faculty Development Coordinator (FDC) and a group member with research experience designed the current programme as a faculty development project. The participants were community-based physicians dedicated to teaching family practice residents. The goal was to involve a group of family physicians in an audit though a shared learning experience employing a physician-friendly, low-cost process. The programme was called ‘Quality assurance through reflective change’ (QuARC). This paper outlines the programme and the reflections generated.

Programme

Participants in this programme were the faculty of the family medicine community-based residency training (CBRT) at McMaster University. These supervisors meet four times a year for faculty development (FDM). The FDC (LE) and a group member (GS) planned to use these sessions to involve preceptors in QA. Fifteen minutes were allotted at each FDM. The first session introduced the project and encouraged involvement in the audit. The second session was designed to reflect on the data-gathering process, on the ongoing feed-back process and on individual and group informa-tion collected to date. Each physician was encouraged to make changes based on personal reflection. The third session was designed to review changes made, to encourage continuation of data gathering and to consider strategies to improve the process. The fourth session was designed to reflect on the full year-long process.

To foster a sense of ownership in and enthusiasm for the process, it was important that the group, consistent with the adult learner paradigm, was involved in deciding on the audit topic. Discussion was based on several evidence-based recommendations from the Canadian Task Force on Preventive Health Care.[2] Mammography rates with in the past 24 months for women aged 50–69 years was agreed upon as a topic of our audit.

The protocol called for collecting data each week on up to 20 consecutive eligible women who had an appointment booked. Each physician was provided with data collection forms to document patient in-itials, age, whether there was a mammogram report within the previous 24 months in the medical record and, if so, the date of the report. Space was provided for any comments thought to be important. Forms were to be sent weekly to the collection centre by fax, email or regular mail.

Data forms were distributed at the first session, minimising delays in starting data collection and loss of any initial enthusiasm. Collated information was fed back to physicians weekly via email or regular mail. Feedback reports varied, sometimes as comparative screening rates, sometimes as encouragement to con-tinue submitting data, sometimes as comments reported from other participating physicians. Reports were kept to less than one page in length. On occasion, indi-vidualised feedback was sent. All CBRT members were sent feedback whether or not they submitted data or attended sessions.

Eight weeks later, at the next FDM, the participants reflected on the project. Participation rates, mam-mography rates and some of the comments that had been received on weekly data forms were summarised to create a positive background to engage physicians in Discussion. Each participant was asked to jot down on a ‘post-it’ note, positive ideas about the data-collection process. The participants were then grouped in threes to find one common point per group. These points were reported back to the whole group without further Discussion. A similar process collected nega-tive points about the data-collection process. All ‘post-it’ notes were collected at the end of the session.

The participants were also asked to comment about the weekly feedback on ‘post-its’. These were collected without Discussion. Next the group jotted down what surprises they had found. After discussing these com-ments the group was encouraged to reflect on the shared ideas and to individualise changes in practice.

Within a week, comments from the ‘post-its’ and the Discussions were summarised and fed back to group members regardless of whether they had col-lected data or attended sessions. Each physician was again encouraged to look at his/her practice with a view to implementing changes that might improve both the data-collection process and the mammography rates. Further data collection was encouraged to mon-itor changes occurring in mammography rates.

The success of the project was measured by the involvement of physicians – both the numbers who responded and the enthusiasm they showed; the number of patients on whom they collected data; the comments and reflections collected as the study progressed and at each FDM session; the indication of need for change; and mammography rates.

Attendance at the FDM is not mandatory. At the first session, 65% (17/26) of the CBRT preceptors were present, and at the second, 81% (21/26). There were 13/26 (50%) who attended both sessions. By the second session, 14/26 (54%) physicians had submitted data, 11 weekly as stipulated in the protocol and three submitted full practice audits. Four physicians who attended both sessions but did not submit data sheets offered the following explanations: a lack of interest in being involved; misplaced the data entry form and found catching up overwhelming; too heavy a work-load to get the information down on paper; arrived at the meeting late so had missed the purpose of the project. Four physicians who attended only the second meeting indicated an interest in submitting data.

At the initial FDM, Discussion about the topic for the audit was enthusiastic. Within the first week, data had been submitted. By the second FDM, there had been six separate pieces of feedback sent to all the physicians who were members of the CBRT group. The 11 physicians who completed weekly audits col-lected data on 613 patients with a mammography rate of 68%. Three physicians with EMRs, who submitted full practice audits, had data on 401 patients with a mammography rate of 73%.

The varied techniques used for feedback (‘post-its’, small-group Discussion and large-group work) gen-erated a good Discussion and collected many ideas. Twelve physicians indicated that the weekly feedback had been positive and that it encouraged participation by being enthusiastic. Three provided no feedback as they had not been involved with the data collection process.

Comments received by the second meeting, through individual communication on weekly data sheets, through email communication, and in other face-to-face contact could be grouped into the following themes; reaction to the weekly feedback, reaction to the audit process, surprise at the mammography screening rates, concerns about the topic, charting issues, data collection issues (see Box 1).

Box 1: Comments from participants after session 2

By the third session, participation was 16/26 (61%). Data collection had continued to 739 data points for the convenience sample group with an aggregate mammography rate of 68% (502/739) (see Figure 1) and 1004 for five full practice audits with an aggregate mammography rate of 80% (804/1004) (see Figure 2). Comments received at session 4 are summarised in Box 2.

Figure 1: Weekly data collection group

Figure 2: Full practice audit group

Box 2: Comments from participants after session 4

Discussion

The QuARC programme was a success at several levels. The participation rate of 61% was excellent. Even some members of the group who were not at the initial session subsequently collected data. The interest in doing further audits and group projects indicated positive reaction to the project. A follow-up project has been initiated with 73% (19/26) of group mem-bers.

Enthusiasm was evident in the heated Discussion in deciding about the topic. Comments after the first data-collection phase and at the last session further underscored the excitement preceptors felt during the project. One physician indicated that he was so excited that he did a full practice audit the day after the first session. Other physicians who had electronic records and wanted to be part of the group project also did full practice audits.

In 12 weeks, data on over 700 patients were entered into the database from weekly data collection, and data on more that 1000 patients were entered from the full practice audits. This large number of patients from 16 peer practices generated mammography rates that were seen as an acceptable standard of care: ‘[The] study [was] based on reality and real practices’.

Perhaps the most compelling sign of successful was the comment, ‘Before I started this study I thought I was doing a good job’. This was a very brave obser-vation reflecting the impact of reviewing one’s own charts.

Physicians saw the audit as a ‘good exercise’ that made them better role models for the residents.

The audit stimulated physicians to question their own charting. Though measuring behavioural change was beyond the scope of this project, this is the first step in generating change. Despite the ready avail-ability of the charts and the minimal amount of data collected, several participants noted it took extra time and was hard to remember to do. This contributed to the range of patients entered by each practice. Yet each participant managed to collect data over several weeks. The lack of a recall process was noted.

There are many factors that contributed to QuARC being a success. The convenience sampling approach and the collecting of only four data points was unique. The concept of doing an audit has been discussed with the group at earlier sessions, and each physician had had residents do an audit in the practice. QuARC, a change from the usual audit process, ignited an already-kindled idea.

QuARC presented by a group member as ‘our’ project was not a dictum from ‘above’. The mam-mography rates were based on a ‘true’ peer group. Group dynamics at the first session generated an enthusiasm that the physicians took home. Peer pressure in a group ‘captured’ for faculty development, may have coerced some to participate. Yet some physicians chose not to participate in data collection and others, not at the first session, did.

The project leader’s enthusiasm was noted as im-portant in stimulating interest in the audit process. Frequent, positive, encouraging feedback strongly stim-ulated data collection to continue. The convenience sampling allowed participants to collect data right away and receive rapid feedback avoiding loss of momentum through prolonged intervals between these steps. The minimal amount of data required for each patient from charts already pulled for visits, motivated physicians to collect their own data and allowed first-hand experience with their mammography practices. The collaboration of the physicians in collecting data allowed quick accumulation of a large peer-generated dataset and an opportunity to com-pare practices of similar physicians.

Community-based physicians who teach were selec-ted because they are seen as good role models for residents, as demonstrating comprehensive primary care practices and having an interest in teaching. Physicians who take residents expose their patterns of practice to the external scrutiny of a learner, and may be more comfortable with the audit process. Physicians committed to teaching residents have a tie to academia that may have contributed to the success of the project. These characteristics of the participating physicians may make them more recep-tive to an audit process than other community-based physicians.

There are several limitations to this study. Although the participation rate was good, participation in the study was voluntary. Physicians who felt they had poor data may have declined to be involved. Self-reported data are subject to reporting only positive cases. Although 11 physicians reported weekly data, the number of patients enrolled weekly varied amongst the physicians. Differences in patient populations and varying involvements in the audit may have influ-enced this. Some physicians collected data for more than 12 weeks, while others collected data for only a few weeks.

The group sessions, limited to 15 min, put time constraints on Discussion. Consensus on goals for mammography rates was not attempted. Some of the benefit of group dynamics as a motivating force for achieving goals was missed. Short sessions may have prevented all ideas from being presented and shared.

Intention to change is not the same as achieving change. The current study was not able nor was it designed to demonstrate actual changes in practice patterns and mammography rates, as these were not measured. Further audit would be needed to measure changes.

Conclusion

This project has shown that an enthusiastic mentor and a collaborating group of physicians can collect meaningful practice audit data to stimulate reflective behaviour using simple methodology and a con-venience sample of patients. This audit process pro-vides an opportunity for physicians to demonstrate the quality of the care they provide with a minimal level of disturbance to their practices.

References

1. Lemelin J, Hogg W and Baskerville N. Evidence to action: a tailored multifaceted approach to changing family physician practice patterns and improving preventive care. Canadian Medical Association Journal 2001;164:757–63.
2. The Canadian Task Force on Preventive Health Care www.ctfphc.org(accessed 2 June 2006).