Discussion Paper - (2010) Volume 18, Issue 2
Professor of Primary Care
Stephen Campbell PhD
Senior Research Fellow
National Primary Care Research and Development Centre, Manchester, UK
Received date: 21 September 2009; Accepted date: 6 February 2010
This paper explains the reasons and context behind the introduction of the Quality and Outcomes Framework (QOF) in the UK in April 2004. The QOF is a pay-for-performance scheme covering a range of clinical and organisational areas in primary care. In 2004, 52% of the framework related to clinical care, increasing to 66% in 2006 and 70% in 2009. From April 2009, the National Institute for Health and Clinical Excellence (NICE) has led a new process for developing the clinical QOF indicators. Clinical areas are now prioritised by an advisory committee appointed byNICE; theQOFindicators then undergo a formal consensus procedure followed by piloting in representative practices across England. However, what are the attributes of a good QOF indicator and how do these differ from those of a good quality indicator, such as validity and sensitivity to change? This paper describes the concept of ‘QOFability’, which relates to why some areas are, and others are not, prioritised for the QOF. Factors include the prevalence of the clinical issue, the accuracy of data extraction from GP clinical systems, the clarity of diagnosis, the relevance of incentivised actions, the direct attribution to all primary care staff and consideration of any possible unintended consequences of introducing any given indicator. The paper concludes by considering the future direction of the QOF, recommending a focus on creating feasible, valid, reliable and piloted ‘QOFable’ clinical indicators.
indicators, QOFability, Quality and Outcomes Framework
There were no real standards for general practice from the 1950s with GPs acting almost entirely on their own conscience.[1] However, by the late 1990s this had ‘changed utterly’[2] due to multiple policy drivers. Some were negative drivers for quality improvement and patient safety as a result of high profile cases such as the Shipman and Bristol inquiries, whereas others were more positive and were based on the establish-ment of the National Institute for Health and Clinical Excellence (NICE) to provide and promote national guidance on good health and preventing and treating ill health. These changes were underpinned by the antecedent establishment of evidence-based medicine[3] and a new public management[4] with an increased use of control and contractual arrangements to improve public services and professionals perceived as inef-ficient. However, three parallel cultural changes also occurred. First, reducing variation in and improving healthcare quality became a Government priority. This was manifested, for example, in the incentives for immunisation and screening in the 1990 GP contract and in the 1997 New Labour administration’s challenging agenda to monitor and improve quality of care. Second, while performance indicators were once ‘dismissed as the dotty idea of a few enthusiasts’[5] methodological approaches to measuring quality be-came available with the advent of quality indicators during the 1990s.[6–8] Third, while the general practice profession had previously rejected claims that there were variations in quality between doctors, the Good Practice Allowance introduced in 1986,[9] as well as a cultural shift towards accepting that care was too variable and could be improved, brought a correspond-ing realisation that improving quality was an oppor-tunity for income generation.[10] These changes were also a response to demoralisation within the primary care workforce[11] driven by an increasing workload and responsibility for unfunded services, such as diabetes clinics, previously provided in secondary care settings. In 2001, a British Medical Association (BMA) ballot found that 86% of GPs would consider resigning if a new contract could not be secured by the BMA because of the underfunding of primary care (www.bmj.com/cgi/content/extract/322/7299/1381).
The introduction of QOF indicators
The QOF, which was introduced in the UK in April 2004 as part of a new General Medical Services contract for primary care was, in effect, a product of the factors described above. In particular, public disquiet over the quality and safety of healthcare services, the rise of evidence-based medicine, a change in the culture of the profession towards recognising variation in the quality of primary care and serious underfunding of primary health care in the UK com-pared to other countries were the key factors under-pinning its introduction.[10] In these circumstances, professional representatives (the General Practitioners Committee (GPC) of the BMA) were able and willing to negotiate with the UK government to provide ele-ments of primary care through a system of perform-ance related pay. The government was willing to invest up to 20% of the primary care budget, 90% of which was new funding, in order to develop a series of incentivised evidence-based indicators across a range of clinical and organisational areas in primary care. The BMA asked the profession to vote on a new practice-based contract containing multiple elements, including the opportunity to opt out of out-of-hours care responsibility and the introduction of the QOF. Seventy percent of the profession voted and of those 79% voted ‘yes’ to the new contract. However, there remains an interesting and unresolved tension at the heart of the QOF – is it a mechanism for paying GPs or rewarding the attainment of quality targets or is it a quality improvement tool?The QOF originally consisted of 146 indicators. The majority of the indicators (76, representing 52% of the framework) were focused on clinical areas, although the use of a ‘balanced score card’ approach is reflected in the mix of clinical, organisational and patient focused elements to the framework (see Box 1). Points for individual indicators were awarded in relation to the level of achievement of that indicator (e.g. the percentage of people with diabetes having blood pressure below a defined target), with a graduated scale of payments that started when a minimum threshold (25% initially but raised to 40% in 2006) and ended once a maximum threshold (usually 90%) was reached.
Since 2004, the QOF has gone through one major (2006) and one minor (2009) reorganisation. In 2006, seven new clinical domains were added (depression, atrial fibrillation, chronic kidney disease, dementia, obesity, palliative care and learning disability) and the number of clinical points was increased to 655 (66% of the total points) within a slightly reduced overall framework of 1000 points. In 2009, the main changes were the addition of a new area of primary prevention for heart disease, making the clinical indicators worth 697 points or 70% of the Framework. Three new sexual health indicators were added to additional services and changes were made to patient experience gathering so that data was collected through a new national survey. Points for the changes made since 2006 were largely released by the removal of a number of organisational indicators and a change to the depth of quality measures.
Although the QOF is a voluntary system, 99% of UK practices participate. During the first year, the levels of achievement exceeded those anticipated by the government, with an average of 83.4% of the available incentive payments claimed.[12] Achievements have increased in subsequent years, with a very slight fall in 2008/9 largely due to changes in the patient experience domain (www.ic.nhs.uk/webfiles/QOF/2008–09/QOF%20Achievement%20and%20Prevalence%20Bulletin%202008–09.pdf).
The Expert Panel process (2005– 2009)
From 2005 to 2009, new indicators in each QOF area were developed by a group of appointed primary care academic experts (the Expert Panel), supported by a group of clinicians who also had an interest in that area. The topics for development came from two ‘calls for evidence’ in 2005 and 2007, which were widely distributed by primary care trusts, the BMA, National Health Service employers and voluntary groups such as the Long Term Conditions Alliance. Over 500 topic area ideas were submitted by a wide range of stake-holders including individuals, patient groups, pro-fessionals, charities, NICE, the Department of Health and the pharmaceutical industry (who were asked to declare any financial interest in their submission) (see Figure 1).
Figure 1: Range of groups submitting ideas in 2007 (n=153)
These ideas were then prioritised by the Depart-ment of Health and the GPC and up-to-date evidence on the selected areas was reviewed. In 2007, two new elements were introduced into the development pro-cess. First, meetings were set up with 20 groups who had submitted prioritised ideas to ensure that the Expert Panel primary care academics understood the intention behind the suggestions and also to explain to groups why some ideas were not suitable for inclusion in the QOF. This reflected a perception that whilst some newspapers were interested in informing the public about the financial implications of the QOF[13] (and indeed most GPs received an increase in their earnings of up to 25% in the first year of the QOF), there was far less information available to the general public about the purpose of QOF or why only certain conditions were prioritised. Second, a modified Delphi procedure[14] was included in the development process to combine available evidence with expert professional opinion. This enabled the development of indicators where evidence was patchy or inconclusive.[15]
Indicators were then commented on by a national patient organisation and by general practice clinical systems experts in terms of their feasibility for im-plementation in primary care. The final set of published QOF indicators represented a negotiated compromise between the Department of Health, who needed to ensure the best possible use of Treasury resources for patient benefit, and the British Medical Association, representing the views and interests of the medical profession.
The new process under NICE
From April 2009, there has been a new NICE led process developing the QOF. Whilst it is not the aim of this paper to describe the new process in detail, there have been a series of important changes that should improve the quality of future indicators. At a macro level, the fact that NICE is an independent organisation should mean that political pressures to include particular topics or types of measure can be examined in a neutral manner. NICE also has a well deserved reputation as a transparent organisation, with committees open to the public and relevant documents available on the web (www.nice.org.uk/aboutnice/qof/qof.jsp)which should reduce some ofthe previous mystique around the process of devel-oping the QOF.
Clinical areas and evidence-based statements within them are now prioritised by the 30-strong NICE ap-pointed Advisory Committee. These then go through a two-stage modified RAND Appropriateness Method[14] including area experts and front-line GPs, through which they are rated for their necessity. Indicators are also reviewed at this early stage by specialists in general practice clinical systems, who can comment on feasi-bility, request Read codes and start to work on the underpinning business rules. Perhaps most import-antly, the indicators that come through the consensus process (approximately 40–50%) are now piloted in 30 representative practices across England and in a smaller number of practices in the devolved nations of Wales, Scotland and Northern Ireland. This means that the final indicators presented to the negotiators will have been tested in the real world. In each pilot the indicators will be tested for feasibility and reliability of data extraction across all general practice clinical systems and data on workload will feed into the cost effectiveness analysis of each indicator. Equally im-portantly, qualitative interviews will be undertaken with general practice staff and patients to ask them their opinion of the value of each piloted indicator in order to place the opinions of staff and patients at the heart of the piloting process. This process is, however, time consuming and the first indicators from the new NICE led process will not become part of the QOF until April 2011.
What makes a good QOF indicator (QOFability)?
deally a QOF indicator like any other quality indi-cator should adhere to some key characteristics, such as validity and sensitivity to change, and achievement against the indicator should be attributable directly to the primary care teams being assessed (see Box 2).[14] However, within the context of the QOF, there are a number of additional ‘QOFability’ issues that each potential area and indicator needs to pass before it can be considered for potential introduction into the QOF. First and foremost, a clinical area has to be common but also important in terms of morbidity and, to some extent, mortality. Otitis media, for example, is ex-tremely common but the significant associated mor-bidity is relatively small, which means it is unlikely to ever become part of the QOF. Multiple sclerosis has very significant morbidity but is relatively rare. Although an average practice of 6300 patients might expect to have about eight people with multiple sclerosis on its list, a single-handed practice might only have one or two patients. Multiple sclerosis is therefore also unlikely to become part of the QOF.
Each QOF domain also needs to be internally coherent, with indicators in a logical order. If only one or two elements of a potential suite of indicators can be made to work within the information tech-nology confines of the QOF, then there is less value in introducing the domain.
From a general practice clinical systems perspective, indicators also need to be unambiguous, able to be extracted in a clear, sequential and consistent manner from a range of GP computer systems through the central Quality Management and Analysis System (QMAS). This partially accounts for why many QOF indicators are single disease and single issue in their focus Indicators within the QOF should also be evidence rather than policy based. All submissions during the previous Expert Panel process needed to state clearly the evidence base underpinning the idea and almost all of the current QOF indicators are evidence based. In the new NICE led process, all indicators are under-pinned by NICE or SIGN guidance, with an aspiration in 2010 to also use NHS Evidence (www.evidence.nhs.uk)as a source.
To be truly QOFable, a condition also has to be something that can be clearly defined and diagnosed. A condition such as osteoarthritis of the knee – a common condition with significant morbidity that affects 12% of people over the age of 65[16] – can be di&ffcult to diagnose consistently, and a diagnosis is usually made on clinical grounds. There is potential for significant variation in prevalence (and therefore GP payments) because of different GPs’ subjective opinions on whether someone has osteoarthritis of the knee or not. The QOF could incentivise a referral for X-ray confirmation, but that has implications for secondary care radiology services (see below) and con-flicts with current radiology guidelines.[17] There are a number of conditions therefore that might seem at face value to be ideal candidates for the QOF, but that fall at the fence of diagnostic ambiguity.
A condition, or indicator related to it, also has to be something that every primary care team in the land could address. An indicator that requires referral for a scan that is not uniformly available to every practice in the UK would not be QOFable. This has been a problem when trying to develop an osteoporosis domain for the QOF. Osteoporosis is a prevalent condition, with significant morbidity and mortality,[18] but access to bone densitometry (DEXA) scans is not yet univer-sal. Whilst not all osteoporosis indicators rely on DEXA scan results, this would be an early indicator in any potential domain. There is a counter argument that perhaps the QOF should drive service innovation, but it is di&ffcult to argue that the QOF should be used as a tool to drive significant changes in secondary care provision in a time of financial austerity.
Attribution to primary care is a further critical aspect of a good QOF indicator. Whilst outcome measures are often seen as the ‘gold standard’, process measures are more useful as a measure of performance in primary care. Hard outcomes such as mortality or smoking cessation rates are of course influenced by the care received in primary care, but they often occur long after that care has been given. They may be con-founded by patient lifestyle choices and socio-demo-graphic factors outside the control of primary care staff or by the availability of secondary care services.[19] Case-mix adjustment can, in theory, be used to adjust outcomes for underlying differences in populations.[20] However, there is usually insu&ffcient information in the medical record to facilitate this for primary care populations. Intermediate outcome measures, measures based on scientific evidence which link them to effect-ive outcomes, are perhaps the more useful indicators in primary care. The QOF has a number of such intermediate outcomes, e.g. those focused on lowering blood pressure in people with heart disease where there is evidence that controlling blood pressure is import-ant in improving survival.[21] However, they are more di&ffcult to achieve and represent a greater workload, which is why they have more points attached to them than simple process indicators, and have lower achievement thresholds.
It is also important to be constantly mindful of the unintended consequences of the QOF,[22] perhaps even more so now we have five years experience of devel-oping the framework. For example, if pure outcomes were prioritised, and perhaps smoking cessation rate rather than rate of referral for advice and treatment might be introduced as an indicator. However, we know that smoking rates are higher and cessation rates are lower in practices serving areas of higher depri-vation.[23,24] This might affect practice income and therefore have the unintended consequence of fewer GPs wanting to practise in deprived areas, where health needs are greatest.[25]
There are also strong arguments for QOF indicators to be focused on health inequalities and to be cost effective.[26] Whilst not disagreeing with this, these issues, of themselves, are not critical to whether an area or indicator is QOFable.
QOFability issues are summarised in Box 3.
Future directions for QOF
The QOF, under NICE, is now developing in a more systematic and transparent manner but there are still plenty of challenges ahead. As the population demo- graphy changes, patients are increasingly likely to present with more than one condition.[27,28] Primary care will provide the majority of ongoing care for this growing population within most healthcare systems. There is therefore a need to develop, pilot and validate sets of measures that take into account the number and severity of conditions at an individual level. There are also unresolved tensions around the purpose of the QOF that still impact on national negotiations. If the QOF is now predominantly a quality improvement scheme rather than a payment mechanism, perhaps the focus should be on creating feasible, valid, reliable, QOFable clinical indicators that are piloted, used and removed in a recognised and accepted manner.[29] They could also be aligned with other organisational and patient experience initiatives in a systems based qual-ity improvement strategy. If the UK succeeds with such an approach, it may finally be able to lay claim to Shekelle’s assertion of creating the boldest initiative to improve the quality of primary care ever attempted in the world.[30]
Commissioned; not externally peer reviewed.
The authors lead the external contractor to NICE, which includes developing and piloting new indi-cators for the Quality and Outcomes Framework.
