Discussion Paper - (2010) Volume 18, Issue 2
1Clinical Senior Lecturer in Primary Care, School of Medicine, Health Policy and Practice, University of East Anglia, Norwich, UK
2Post-doctoral Fellow, Department of General Practice and Primary Health Care, Ghent University, Ghent, Belgium
Received date: 29 January 2010; Accepted date: 1 March 2010
BackgroundA new contract between UK primary care practices and government was implemented in April 2004, with substantial financial rewards to general practices for achievement of standards set out in the Quality and Outcomes Framework (QOF). Aim We aimed to review the evidence about the effects of the QOF on health care, including unintended outcomes, and equity. MethodsRelevant papers were identified by searching Medline and from the reference lists of published reviews and papers. A separate systematic literature review was conducted to identify papers with information on the impact of the framework on inequalities. ResultsAll studies were observational, and so it cannot be assumed that any changes were caused by the framework. The results both for individual indicators and from different studies vary substantially. The diverse nature of the research precluded formal synthesis of data from different studies. Achievement of quality standards was high when the contract was introduced, and has risen each year roughly in line with the pre-existing trend. Inequalities in achievement of standards were generally small when the framework was implemented, and most have reduced further since. There is weak evidence that achievement for conditions outside the framework was lower initially, and has neither worsened nor improved since. Some interventions in the framework may be cost-effective. Professionals feel consultations and continuity have suffered to some extent. There is very little research about patients’ views, or about the aspects of general practice not measured, such as caring, context and complexity.ConclusionThe evidence base about the impact of the QOF is growing, but remains patchy and inconclusive. More high quality research is needed to inform decisions about how the framework should change to maximise improvements in health and equity.
epidemiological factors, healthcare disparities, incentive, primary health care, quality of health care, reimbursement
Many people still do not receive health care that has been shown to improve outcomes, even when healthcare systems are well funded and widely available.1,2 A new contract between UK primary care practices and government was implemented in April 2004, as one of theUKgovernment’s quality improvement initiatives. Substantial financial rewards were provided to general practices through the QOF for achievement of standards in four domains; clinical, organisational, additional services and patient experience (Table 1).3 The clinical domain initially contained standards for ten chronic conditions, rising to 20 in 2009 to 2010.4
The scale of reward payments was far greater than previous performance-related payments to UK general practices for immunisations, cervical screening or health promotion clinics, and the new contract was described as a ‘radical experiment’.5 It was an experiment on a very large scale, without a comparison group, and with no planned evaluation, and so the rapidly growing number of publications about the effects of the contract rely on observational evidence and innovative use of existing datasets.
The scale of reward payments was far greater than previous performance-related payments to UK general practices for immunisations, cervical screening or health promotion clinics, and the new contract was described as a ‘radical experiment’.5 It was an experiment on a very large scale, without a comparison group, and with no planned evaluation, and so the rapidly growing number of publications about the effects of the contract rely on observational evidence and innovative use of existing datasets.
Search strategy
This review is limited to the UK QOF and does not consider the wider literature on payment for performance. Included papers relevant to QOF were identified from the reference lists of two recent reviews of the international literature on payment for performance, 6,7,10 as well as from other published papers and authors in the field. A supplementary electronic literature search for more recent papers (up to 26 January 2010) was conducted using Medline. To study the impact of the QOF on inequalities, a separate systematic literature review was conducted using Medline, Embase, the Cochrane Library, Web of Science, PsychInfo and Econlit. Studies with a longitudinal or serial cross-sectional design were included, and the quality of the selected studies was appraised based on the criteria of the Dutch Cochrane Centre.
Most research into the effects of the QOF has concentrated on the clinical domain. Twenty-nine studies examined the impact of the QOF on health care, four reported on the impact on professionals and two reported on costs (see Appendix online at www.radcliffe-oxford.com/journals/J10_Quality_in_ Primary_Care/supplementary_papers.htm). Nearly all relevant studies were observational studies, either cross-sectional, repeated cross-sectional (before and after contract implementation) or cohort studies. Most data had been collected after contract implementation in 2004, but some studies included pre- QOF data.
Analysis of QOF data from all general practices in England showed that the median overall achievement reported by practices was high in year 1 (85.1%, interquartile range 79.0–89.1), and has increased since, to 89.3% in year 2 and 90.8% in year 3.11,12 A median of 5.3% of patients considered unsuitable for the indicator were excluded by practices from their returns.13
Diabetes is the most studied QOF condition. National QOF data show substantial improvements in diabetes care from 2004 to 2008, with the percentage of low-performing practices dropping from 57% to 26%.14 A systematic review found that the quality of care reported in the QOF was higher than that in previously published studies,15 and improvements in quality in diabetes care have been consistently found in studies which used either QOF or non-QOF data. Diabetes care in Scotland improved from 2004 to 2005;16 the Wandsworth Prospective Diabetes Study found improved support for smoking cessation in those with diabetes between 2003 and 2005;17 improvements in performance ranged from 6.6–42.8% in general practices in North Warwickshire from 2004 to 2005,18 and from 9.2–40.9% in Shropshire from 2004 to 2006.19,20 One small hospital-based study reported an increase in referrals for poor glycaemic control.21
Improvements have also been reported for other conditions following the introduction of the contract, using both QOF and alternative data sources. Care for coronary heart disease (CHD), stroke and transient ischaemic attacks (TIAs) and blood pressure, as well as smoking indicators, all improved.22–25
Underlying trends in quality
The improvements described above in recording of care after the introduction of the contract might of course have happened anyway, with or without the QOF. A number of studies have looked at underlying trends in quality of care in an attempt to estimate the effect of the new contract on existing trends. Several conditions had been the subject of major quality improvement initiatives prior to the QOF (see discussion section) and it would be reasonable to expect a background of improving quality in these conditions.
Data from nearly 500 English practices in QRESEARCH for 19 QOF indicators from 2001 to 2006 showed that the percentage achievement of these quality indicators improved in all ages and among men and women, and that in nearly all cases the trend showed a gradual improvement over the five years, with little change around 2004.26 The exceptions to this were for recorded blood pressure below 150/90 mmHg, recorded cholesterol less than 5 mmol/l or blood pressure below 150/90 mmHg in those with stroke, and blood pressure below 140/85 in those with chronic kidney disease, all of which appeared to show a slightly increased improvement from around 2004, greater than the underlying trend. There was also a dramatic improvement in the percentage of those with epilepsy reported free of convulsions, but the report’s authors comment that: ‘the Read codes to record this (epilepsy indicator) were not in general use in general practice prior to April 2003 and so the increase is likely to represent an increase in recording rather than a true increase’.
A cohort study in 42 English general practices measured care for asthma, CHD and diabetes in 1998, 2003 and 2005, and compared 2005 care with that predicted on the basis of the 1998 to 2003 trend.27 A small, significant, above the trend increase in care was found for diabetes and asthma, but no significant difference was found for CHD. This is consistent with a retrospective cohort study of diabetes care in 26 practices in London, which found that the proportion of those with well controlled diabetes (i.e. HbA1c less than 7.4%) had increased each year from 2000 to 2005, and that the 2005 increase was the largest.28
A study of diabetes care in 422 general practices in the General Practice Research Database found that achievement of HbA1c targets was slightly lower than that predicted by the underlying trend, whilst achievement of blood pressure and cholesterol targets was slightly higher.29 A retrospective cohort study of 147 practices found that outcomes for diabetes care (glycaemic control, cholesterol levels and blood pressure) improved steadily from 2002 to 2005, and improvement was then attenuated between 2005 and 2007.30
Achievement of non-incentivised quality indicators
national survey of care in 2004 found that indicator achievement in QOF conditions was 75%, compared with 58% in non-QOF conditions.1 A study comparing changes in care for incentivised and nonincentivised conditions from 2003 to 2005 found that baseline care for non-incentivised conditions was much lower (35%) than for incentivised conditions (75%), and this did not change with the introduction of the contract.31 This finding may reflect the fact that the non-incentivised conditions had received less policy attention prior to QOF, and there may not have been an underlying trend towards improvement in these conditions. A study of prescribing data from Scotland also looked at non-QOF interventions, and found that the prescribing of QOF drugs increased significantly faster than that of non-QOF drugs both before and after the introduction of the new contract.32
Most studies show little relationship between QOF achievement and health service activity or health outcomes. Associations between QOF scores for six conditions and emergency admissions and mortality were small and inconsistent;33 no association was found between QOF CHD scores and CHD admissions in all primary care trusts in England,34 and no associations were found betweenQOF stroke scores and adherence to stroke guidance from the Royal College of Physicians 35 or between QOF achievement and adherence to British Thoracic Society spirometry standards.36 However, there was a significant positive relationship between QOF scores for epilepsy seizure-free patients and epilepsy related emergency hospitalisation.37
A modelling study estimated that the potential reduction in mortality from full implementation of the contract might have been 416 per 100 000 people per year in 2004 to 2005, and 451 in 2006 to 2007. In 2004 to 2005 the potential reduction in mortality per 100 000 people per year ranged from 163 in coronary heart disease to eight in asthma.38 Importantly these numbers did not take account of any pre-contract activity, so the actual gain from the contract implementation would have been considerably less. An earlier model of the potential benefits of treating cardiovascular disease in QOF allowed for baseline treatment, and estimated that 29 events per 10 000 people per five years could be prevented by reaching cholesterol reduction targets (15 in CHD, seven in stroke and seven in diabetes).39 An additional 15 events could be prevented by achieving blood pressure control targets in hypertensive patients.
Impact on professionals
There has been concern that an unintended consequence of the QOF might be to reduce the professionalism and internal motivation of doctors, and so crowd out the caring aspect of consultations.40,41 There is no evidence that the internal motivation of general practitioners has been damaged,42 although doctors were more enthusiastic about targets which were aligned with professional priorities.43,44 Both doctors and nurses have reported concerns about the emergence of a dual agenda in consultations, with less time for holistic care, patients’ concerns and non-incentivised care and a perceived loss of continuity of care.43–45 They were also concerned about increasing use of templates or ‘box-ticking’ to manage performance.42–45
Cost consequences
A study comparing estimated mortality gain from eight preventive interventions with the estimated QOF payments for those interventions concluded that there was no relationship between pay and health gain.46 A larger study by the University of York identified costeffectiveness evidence for 12 indicators in the 2006 revised contract with direct therapeutic effect.47 The three most cost-effective indicators were use of ACE inhibitors or angiotensin receptor blockers for chronic kidney disease, anticoagulant therapy for atrial fibrillation and beta-blockers for coronary heart disease. The per-patient payment in 2004 to 2005 ranged from £0.13 (for a chronic kidney disease indicator) to £87.79 (for a mental health indicator). The cost-effectiveness of an indicator varied by its baseline achievement, with generally smaller changes needed for an indicator to be cost-effective at low baseline uptake than at higher baselines. The authors urged caution in interpreting their results for a number of reasons, particularly the major uncertainties in the cost and qualityadjusted life year evidence, and uncertainty about the generalisability of estimates to the UK.
Impact on equity
Health inequalities are a major policy and public health concern.48 Six papers describing studies with a longitudinal or serial cross-sectional design with points of measurement before and after April 2004 were identified.17,24,49–52 Four additional papers describing serial cross-sectional studies were included.12,16,22,23 Before the introduction of the QOF, the evidence about inequalities in health care was mixed. Achievement of some indicators was lower for some groups, for example for older patients with coronary heart disease,22 and for female and older patients with a recording of stroke or TIA.23 However, there were no significant differences in care in other areas (for example smoking cessation advice rates in deprived and affluent groups17) and sometimes better care was recorded for minority groups (such as better monitoring of blood pressure in black patients).50
As discussed above, quality of care generally improved over time around the introduction of the QOF, and whilst all groups benefitted from this improvement the relative rate of improvement differed between groups. The resulting changes in inequalities over time are small, variable and dependent on the indicator, the level of achievement before QOF and the demographic variable (age, sex, socio-economic status or ethnicity). However, some general patterns can be identified.
First, the gaps in care between different age groups for CHD, diabetes and cerebrovascular disease (CVD) were attenuated after the introduction of the QOF due to greater improvement in the worse off. For quality indicators with lower achievement for older than younger people, in the three studies from Scotland there were greater improvements for older people in some but not all indicators.16,22,23 For the minority of indicators in which achievement was higher in older people than younger before the QOF, the advantage either persisted16,23 or disappeared because of the levelling up of care for younger groups.17
Second, the gaps between achievement scores for men and women persisted or sometimes increased. Before the introduction of the contract, the studies from Scotland report that achievement scores were significantly higher for men than women for two of eight diabetes indicators, seven of 11 CHD indicators and seven of nine CVD indicators.16,22,23 After the QOF, higher achievement for men was found again for nearly all these indicators and for an additional two of 11 CHD and three of eight diabetes indicators.16,22,23 However, care improved more for women than men in one CHD indicator22 and for recording of the smoking status of diabetic patients.17
Third, the gaps in achievement between the most and least deprived areas have been attenuated in England. The results differ between the studies of 310 practices in Scotland and those using national data in England. In England, the existing gap in the first year after the introduction of the QOF narrowed and almost disappeared in the following years.12,24,51 Interestingly, improvements in achievement were associated with worse previous practice performance, and not with area deprivation.12 Nevertheless, in individual indicators large differences remain (for example, for five of the 147 measured QOF quality indicators the difference between the least and most deprived areas is larger than 10%51), and the poorest performing practices remain concentrated in the most deprived areas.12
Results from a smaller dataset of 310 practices in Scotland show a slightly different picture. Before the QOF, lower achievement was found in more deprived areas for a relatively small number of quality indicators related to CHD, diabetes and CVD (e.g. one of 11 indicators for CHD), and for some indicators achievement was higher in more deprived areas (e.g. achieving cholesterol targets).16,22,23 After the introduction of the QOF achievement of some indicators improved less in the most deprived areas (e.g. an additional inequity for three of nine CVD indicators), leading to a bigger inequality in care.23
Fourth, the impact of the implementation of the QOF differs by ethnic group. Both before and after the implementation of the QOF the results regarding ethnic differences have been variable. Studies focus mainly on CHD and diabetes. Before the QOF, South Asian CHD patients had better controlled cholesterol than white and black patients, and afterwards they scored better in three additional aspects of care. The gaps in CHD achievement between black and white people reduced after the implementation of the QOF in some but not all indicators.52 Pre-QOF variations in achievement of diabetes indicators between ethnic groups were not attenuated in 2005.49
Summary of impact of the QOF
The reported achievement of quality indicators in the QOF was high in the first year that the contract was introduced, and has risen further each year since. These improvements have taken place against a background trend of improving quality of primary care for many conditions in the QOF. Beyond that, it is difficult to draw firm conclusions about the effects of the QOF, despite the increasing number of publications. There have been no evaluations of QOF using an experimental design, and the observational approach used in all the studies reported here limits any conclusions that can be drawn about whether the QOF caused any observed associated changes in outcomes. The diverse nature of the research means that it is not possible or desirable to formally synthesise the data from different studies to produce an overall aggregated result about the impact of the QOF.
Bearing these caveats in mind, a tentative conclusion, in need of confirmation from more research, is that the research reviewed here shows that performance after the QOF has been roughly in line with the trend predicted from the years before the QOF, and at least some QOF interventions are probably cost-effective and equity stimulating. However, there is much variability at individual indicator level. Professionals feel consultations and continuity have suffered to some extent. There is very little research about patients’ views of the impact of the QOF, or about the aspects of general practice not measured in the QOF, such as caring, context and the management of complexity and multiple conditions.
The achievements reported from national QOF data have also been reported from other sources, especially for diabetes, and so it is likely that there have been real gains in quality, rather than simply better recording of existing care.1,26 The evidence about improvements relative to underlying trends is mixed, with some evidence for performance slightly above predicted trend (e.g. meeting blood pressure targets), and some evidence for performance slightly below trend. The mixed results should be expected given the range of conditions studied in different practices, and the uncertainties around modelling trends. It is not known whether pre-2004 upward trends would have continued in the absence of the QOF or tailed off, and if they had continued, whether they would have remained linear as they approached the ceiling of 100% achievement.
The conditions incentivised in the first year of the QOF are different from other non-incentivised conditions in at least two ways. First, they have been wellresearched and are relatively straightforward tomeasure. Second, they have been subject to previous national policy interventions such as national service frameworks 53 and NICE guidelines,54 and so were generally well-managed before the QOF. There is no evidence that non-incentivised conditions were neglected any more after the QOF than they were before.
It is not surprising that almost no links have been found between QOF scores and hospital admissions and mortality, given the wide range of other factors associated with these outcomes and the variable time scales. The exception is epilepsy, where the link between the QOF (epilepsy seizure-free patients) and the outcome (epilepsy related emergency hospitalisation) is reasonably tight.
The qualitative studies of professionals broadly agree that dealing with the QOF has taken some attention away from dealing with patients’ concerns, and continuity of care may have suffered. And what about cost? The QOF has been very expensive, and the single study of cost-effectiveness cautiously offers some reassurance that it may have been worthwhile, at least for the 12 indicators with direct therapeutic effect studied. This has some support from evidence that a diabetes pay-for-performance programme delivered a return on investment, albeit in a very different context in the USA.55
The lack of equity related theoretical and conceptual rigour in the studies, and important methodological limitations such as the lack of data on nonusers of the healthcare system, make assessing the equity dimension of the QOF very difficult. There are a number of possible explanations for the more consistent picture of reduction in inequalities by area deprivation seen in the national English data than in the Scottish studies. The results may be sensitive to the measure of quality used,56 or the subset of practices studied may be different from other practices nationally. In England the findings from local and national studies are inconsistent, with local studies of QOF indicators and deprivation reporting either no association between quality and deprivation, or better quality in more deprived areas.57–60
Good quality health care should be effective, efficient, safe, timely, equitable and patient-centred61 and, as noted above, the biggest gap in the research reviewed here is in patients’ and users’ views of the QOF. There is also a need for more research into the nonincentivised aspects of the QOF, which in turn requires the development of better measures for those important aspects of care that are currentlyunmeasurable. Studies are needed that address the implementation of the QOF from a broader theoretical perspective of equity, including distinguishing between horizontal and vertical equity, and between equal access, treatment and outcomes for equal need.62 There is a need for databases linking individual data over the years, rather than serial cross-sectional studies, and for individual rather than area-based measures of socioeconomic status.
There is uncertainty about the organisation of the QOF: how big should the financial rewards be? How low or high should achievement thresholds for payment be set, given that baseline achievement for many indicators was already above the thresholds for maximum payment,30 and that non-eligible patients can be excepted from indicators?63 What proportion of the total incentive payment should be attached to patient measures of quality, rather than self-reported data?64 How should conditions be chosen for inclusion in, and retirement from, the QOF? What will happen to performance after an indicator is retired? Should the size of reward be determined by potential population health gain, or by the workload for primary care professionals?
Evaluating the impact of a major policy initiative such as the QOF is important in order to inform future health policy decisions. The tentative conclusions of this paper require further research to strengthen or refute them, but equally important is a forwardlooking research agenda. The QOF is likely to be with us for some time to come, and will be managed by NICE.8 NICE will have to decide on developments to the clinical and health improvement indicators for the QOF whether or not evidence is available, and this review has suggested some areas where future research efforts could usefully be targeted.
Commissioned; not externally Peer Reviewed.
None.
