Discussion Paper - (2010) Volume 18, Issue 2
Clinical Senior Lecturer
Maria Kordowicz BSC (Hons) MRes MSc
PhD Student
Department of Primary Care and Public Health Sciences, King’s College London School of Medicine, London, UK
Received date: 22 December 2009; Accepted date: 13 February 2010
exception reporting, gaming, general practice, primary care, quality improvement, Quality Outcomes Framework
The QOF has in many ways been a triumph. A triumph of hope, because hopes have been high about the potential of the QOF to promote the quality agenda in primary care; and a triumph of expectation, as this revolutionary change in working practices for primary care teams has been incorporated into the everyday routine of practice life. It is hard now to imagine consultations with patients and strategies for practice management without the ever present spectre of the QOF. Whether it is QOF ‘alerts’ (reminders) appearing unbidden on the computer screen during the consultation, motivational presentations about QOF targets within reach, or the more general accept-ance in primary care that the QOF strengthens our public health role, it seems that QOF has succeeded in becoming part of the fabric of general practice. Yet to what extent do these apparent successes of the QOF merely represent a smokescreen masking the real picture within primary care? Despite its tangible successes, assumptions about the QOF’s power to capture and improve quality in primary care need to be revisited.
The performance of primary care against the QOF proved to be far higher than was expected at the time the QOF was introduced.[1] The Department of Health based calculations around pay on an expected QOF score of 750 at the end of the first year (2004–2005). In fact, the 8600 general practices in England had a mean QOF score of 958.7 (out of a maximum possible score of 1050 points) which represented 91.3% of available points.[2] Two hundred and twenty-two (2.6%) of these practices achieved the maximum score. In spite of several annual revisions to the QOF, revising targets upwards and adding indicators, the 2008 to 2009 mean QOF score achievement was 954.2 of available points (the maximum is currently 1000 points) with 2.0% of practices achieving the maximum score.[3]
Achievement of high QOF scores brought with it higher performance-related pay than expected. Higher pay and a sense of professional pride have translated into better morale for general practitioners (GPs).[4] This, in turn, has offered some easing to the recruit-ment crisis of the early 2000s, when international recruitment drives in Europe seemed to be the only way of filling GP vacancies.
These gains have also translated into public health gains, albeit on a rather piecemeal basis. The weighting of QOF points, since they reflect a pay deal for GPs, is driven by the assumed workload attached to achieving each indicator and not by the likely benefit to patients. Thus, for example, the indicator DM23 (at least 50% achievement of an HbA1c target of 7.0 or less for diabetics) is awarded 17 points whereas DM18 (influ-enza vaccination target of 85% for diabetics) merely attracts three points. Moreover, many of the public health indicators within the QOF such as blood pressure, cholesterol and HbA1c control were improving before the arrival of the QOF. Nevertheless, there has been evidence of public health gain, with substantial im-provements in, for instance, the management of diabetes and control of cardiovascular risk factors in diabetic patients.[5]
A further public health success has been the drive to reduce health inequalities. The differences in QOF achievement between deprived and prosperous areas have been small and, over time, there is evidence that these differences have diminished.[6] The narrowing of target differences between rich and poor communities has been part of an overall trend of improved per-formance, with slightly greater improvements seen in more deprived communities.[7]
There can be no doubt about the concrete improve-ments since the QOF was instituted in terms of overall quality improvement and ‘intermediate outcomes’ such as blood pressure and cholesterol control. However, the successes of QOF have been tempered by concerns that some of the achievements might not be as sub-stantial as they appear.
One reason for questioning the success of P4P in its incarnation as QOF is that three technical features of QOF may have diminished the reach of performance targets.
First, the process of ‘exception reporting’ necess-arily allows certain patients, deemed ‘unsuitable’, to be excluded from the overall target for patients registered at the practice.[8] Patients may understandably be excluded if they are terminally ill or if they do not agree (after three written requests) to attend an appointment at the surgery for the management of their chronic disease. The overall exception reporting rate for 2008 to 2009 was 6.88% for indicators measuring an outcome and 1.70% for indicators measuring a process. So, on average, almost 7% of patients in England are excluded from public health targets such as achievement of a serum cholesterol of < 5mmol/l.
Second, the targets are not set at 100%. Again this is understandable given the practical difficulties of achieving clinical targets. These targets are rarely achieved in research trial conditions, let alone in routine practice, even with often large financial incentives to spur on the team. However, targets set at 70% for blood pressure control or cholesterol control in cor-onary heart disease (CHD6 and CHD8) exclude 30% of patients from these public health targets. Thus, in combination with exception reporting, targets set below 100% may shift the focus of the practice away from harder to reach patients, in exchange for more efficient achievement of Results.
Third, the prevalence of each of the 19 chronic diseases currently included in the QOF is not inde-pendently verified. A practice may simply have lacked vigour in building up their disease registers; patients who, for one reason or another, have not been coded or have been incorrectly coded will not be on the disease register. They will therefore be invisible to QOF targets and again the public health effectiveness of population targets will be further reduced.
The success of the QOF may be tempered in other respects. Performance may have improved in domains covered by performance indicators but remain static in areas out of the spotlight, such as rheumatological and gastrointestinal disease. Increasingly, there are sug-gestions that performance against current criteria has now reached a ceiling and that other approaches are neededto coaxfurtherimprovementsoutofprimarycare.
The subject of gaming and manipulation of target achievement is controversial. Gaming is not unique to the QOF and is probably a feature of all P4P systems. The National Audit Office report on the 2004 contract for GPs suggested that QOF income could be inappro-priately boosted by deliberately removing patients from disease registers or by increasing levels of excep-tion reporting.[9] Although all general practices are given an inspection-type visit annually by representa-tives from their primary care trust, this may be insuffi-cient to detect evidence at case level of inappropriate exception reporting or exclusion from disease regis-ters.
Gaming may generate overlarge financial rewards in just a few practices. But how widespread a phenom-enon is it? Some have suggested that gaming is endemic, but a more balanced perspective has emerged from the Centre of Health Economics which concluded that practices could have treated 12.5% fewer patients without falling below upper QOF thresholds.[10] This suggests that GPs have not taken the opportunity to produce a threshold gaming effect, whereby the quan-tity and quality of work can be reduced to the min-imum needed to meet the target. In other words, GP practices had overshot targets to a much greater extent than the likely level of exception reporting.
Practices with more highly developed management infrastructures and a shared ethos of coding every possible QOF-related activity will inevitably have higher QOF scores at the end of the accounting year. Many apparent improvements in care amount to little more than increased conscientious coding. For example, a practice failing to reach the 90% target for retin-opathy screening in diabetes (DM21) may find that this target is achievable simply by searching through scanned correspondence from the hospital diabetic clinic or local optometrist reported retinopathy findings.
Practices may be making economic decisions based on workload, time and the type of professional needed to reach the target. On this basis, a practice may make one of three decisions. It may decide that it is not cost effective to chase the final QOF point (achieving the 90% target for DM21 is worth five points) and remain below the top target. Or it may invest in additional data input staff to find and code missing clinical data. Or, and most expensively, it may invest in additional medical personnel to examine, say, an additional ten diabetic patients in order to gain all five available QOF points.
It is these pragmatic decisions, based on percep-tions of workload and reward, that have resulted in some commentators describing the QOF as not so much a P4P system, but a ‘pay-for-reporting’ system.[11] Better recording undoubtedly Results in higher QOF points but arguably may not represent better care. Equally, low scoring practices may be less skilled at handling large data volumes and may not necessarily be providing poorer care. The care provided by low scoring practices has not been evaluated in Peer Reviewed studies and we need research information on whether these practices do offer high quality care (which is inadequately recorded) or whether care falls below acceptable standards.
The QOF can only measure a small proportion of all primary care or GP activity and it is possible that low scoring practices display excellence in other domains not captured by QOF, such as continuity of care, patient-centred consultation skills, diagnostic skills and the care of illnesses not included in the QOF.
Increasingly, proposals for strengthening the QOF are focusing on aligning the indicators and the associated QOF points with public health gains. Now that NICE has taken overall responsibility for QOF development, the expectation is that the QOF will develop along the lines of NICE guidelines and continue to favour clinical indicators with a strong evidence base.
In an evaluation of the QOF in its original incar-nation, Fleetcroft et al concluded that there was ‘no relationship between pay and health gain’, at least for the eight public health and preventative interventions which were included in their study.[12] This is perhaps unsurprising because the level of P4P financial reward was based on estimates of likely GP workload rather than on health gain for patients. However, if QOF continues to be generously funded, it has to be able to demonstrate that the money is well spent given that the opportunity costs of tying up healthcare Funding in the QOF are considerable.
Professionalism is one driver of quality which is in danger of being ignored by the QOF. And yet it is a sense of professionalism, the accumulation of a body of specialist knowledge and wisdom placed at the service of society, and a public service ethos which in the longer term probably motivate GPs more than a financially driven P4P system. It is hard to see how any P4P system could reward the components of pro-fessionalism, as expounded by Downie.[13]
Finally, greater inclusion of feedback from patients in the overall spread of P4P indicators is being explored. Prior to 2008 to 2009, GPs simply gave out questionnaires to a selection of patients and ‘patient experience’ points were awarded on the basis of com-pleted surveys and reflection on the Results of these surveys. Since then a more rigorous approach has been adopted, with independent patient surveys conducted by polling organisations. GPs are now rewarded on the basis of responses to two questions about time taken to access an appropriate health professional (PE7 and PE8). The questions asked have been criticised as being politically driven; they do not ask about the consul-tation or perceived standards of care. Yet their in-clusion does mark a new emphasis on rewarding GPs on the basis of patient feedback. One danger of this approach is the unintended consequence of less satis-fied responses from patients in deprived communities which may result in more Funding being directed toward practices serving populations with lower healthcare needs.
So is the QOF all smoke and mirrors, or has it produced real improvements in patient care? We have discussed evidence of its successes in the man-agement of long-term conditions and of improved financial rewards linked to GP workload. However, we have also shown that the QOF is not immune from gaming behaviours and the opportunity for data manipulation through, for instance, reporting as ex-ceptions patients who are harder to engage. Bold statements about the QOF’s power to reduce health inequalities ignore the subtleties of organisational behaviour change in the face of financial rewards. There is no doubt that the underlying essence of good primary care lies in soft data and therefore is unlikely ever to be fully captured through P4P initiat-ives – rapport, interpersonal skills, continuity of care (passion and compassion, even!) to name but a few. Yet with greater patient involvement in the primary care agenda, there is likely to be a move beyond QOF based targets to targets based on indicators which hold meaning for the patient. The challenge will be to ensure that these new indicators do not become a smokescreen.
Commissioned; not externally Peer Reviewed.
Competing interests: My income is directly linked to my QOF performance (MA).
None declared (MK).
