Quality in Primary Care Open Access

Keywords

diabetes mellitus, information aims, quality domain, quality indicator, stakeholder preferences

Background

Transparency in health care has become essential for healthcare consumers, healthcare providers, purchasers and policy makers. These stakeholders may use quality information for different purposes, including internal quality improvement, cost containment and accountability. [1,2] Currently, a great deal of information on quality has become available but is not optimally used.[1,3–5] This information is mostly expressed through quality indicators, beingmeasurable elements of healthcare performance. The presentation format of quality indicators can be a barrier to their use by healthcare consumers.[1,6,7] Another barrier is that physicians may doubt the accuracy and relevance of these quality indicators.[8] Ideally, quality indicators are developed in consensus procedures, such as Delphi techniques, in which stakeholders are extensively involved and heard.[9] Before arriving at a consensus on specific indicators, it is essential to learn more about stakeholders’ perspectives on quality indicators. Without this knowledge, one could easily develop a set of public indicators that does not adequately fit with the aims of the different stakeholders, thus creating a barrier to their implementation.[10] Despite the importance of such knowledge,[11,12] there is little research in this area.

We therefore conducted a qualitative study to explore stakeholder preferences for quality domains in relation to the intended use of quality indicators. To compare these preferences, we felt it important to explore stakeholders’ interpretation of the quality domains. We hypothesised that different stakeholders with different interests may have different interpretations of quality domains and therefore different preferences. We restricted our study to primary diabetes care because several quality indicators for diabetes care are already available, although a national set was still lacking. We used the framework for quality assessment from the Institute of Medicine (IOM).[13]

We also explored preferences for structure, process, or outcome quality indicators for diabetes care.[14]

Methods

Study population

The present study draws on 14 semi-structured interviews with people from seven different organisations involved in the assessment of diabetes care quality using quality indicators. We aimed to include professionals who had a decisive influence on the inclusion of quality indicators in a national indicator set. This is not a very large group of people in the Netherlands. The interviewees represented: (1) healthcare consumers, (2) purchasers, (3) healthcare providers and (4) policy makers (Table 1). Purposive sampling was used to identify senior staff members who were engaged in quality of care measurement and improvement using quality indicators. For healthcare consumer perspectives, we contacted the federation of patient and consumer organisations, because this organisation deals with patient rights and preferences in theNetherlands.Purchasers’ perspectives were covered by the three main health insurance companies. For a healthcare provider perspective, we selected active members (diabetes nurses, primary care physicians and medical specialists) of the different professional organisations involved in healthcare quality improvement (Table 1). For policy makers’ perspectives, we contacted the health inspectorate, the authority within the Ministry of Health,Welfare and Sports responsible for the quality and safety of health care provided in the Netherlands. All participants received a letter containing information about the aim and methodology of the study in advance.

Interview guide

The interview guide was based on the quality framework of the IOM,[13] comprising quality domains of safety, effectiveness, patient-centredness and timeliness. According to the IOM,[15] the vast majority of quality indicators address effectiveness and safety, a smaller number examine timeliness and patient-centredness, and very few indicators assess the efficiency or equity of care. The IOM found that aspects of efficiency are reflected in other components of quality, with the exception of cost per unit service.[13] There is also considerable confusion on how efficiency is defined: sometimes re-admission rates, procedure rates or costeffectiveness have been considered to be efficiency indicators. McGlynn’s report on the definition of efficiency[16] defines efficiency as a relationship between a specific product (output) of health care and the resources (input) used to create the product. We left out efficiency because of this misunderstanding and its incorporation in other components. With regard to equity of care, the IOM stated that the four domains could differ between different groups in the population or across geographic areas,[13] and therefore equity was not considered as a separate quality domain.

We used Donabedian’s model of quality measures addressing three levels of health care and its intended results: structure, process and outcome.[17] A semistructured guide (Appendix) was used, including openended questions on: (1) the aims of using information on quality in order to clarify stakeholder perspectives on such information; (2) the interpretation of and preferences for the quality domains in relation to specific user aims, with a possibility of submitting additional domains; and (3) the preferences for information on quality in terms of structure, process and outcome, again in relation to specific user aims. The interview guide was pilot tested prior to the data collection.

Data collection and analysis

Fourteen face-to-face interviews were held between July and December 2009. All participants gave written consent to participate in the interviews. Interviews lasted between one and two hours. The interviews were conducted by two researchers: one conducted the interviews and the other took notes. All interviews were recorded on digital recorders with the permission of the participants. In order to ensure data accuracy, interviews were translated verbatim by the two researchers present at the interviews. The transcripts were then analysed independently by both researchers and emerging codes were systematically discussed by the research group. In case of disagreement, an independent (third) researcher would check the corresponding part of the transcript against the original interview recording. Any points of disagreement were resolved through discussion by the research group. Finally, summaries of the interviews including the emerging themes and the conclusions were sent back to interviewees, who were asked to check on consistency and accuracy.

The transcripts of each interview were analysed using content analysis18 according to a predetermined framework. This framework consisted of two main themes: the IOM quality domains and the Donabedian approach for quality indicators. Within these predefined themes, we identified parts of text (concepts) that related to categories. Next, we coded data by giving descriptive code names to the concepts (for example, GP, aim ‘internal quality improvement’, category ‘interpretation safety’, concept ‘patient level’). Later, we grouped similar codes under the larger, main concept. Finally, we organised our data by stakeholder in order to identify similarities and differences.

Qualitative analysis software (ATLAS.ti Win 6.1) was used to facilitate organising the data into concepts, categories and themes.19

Results

Aims of using quality indicators

All stakeholders cited ‘improving the quality of care’ as the overarching goal in the use of quality measures. Stakeholders had different, as well as overlapping, aims and the number of aims per stakeholder varied from one to four. Healthcare consumers indicated that their interest in information on quality was for selecting care providers and services. Healthcare providers stated that they used information on quality for internal quality improvements (e.g. through audit, feedback and benchmarking). Policy makers and purchasers indicated that they used information on quality to detect areas of improvement and to inform healthcare consumers. In addition, policy makers aided quality improvement by stimulating cooperation between providers. Purchasers also mentioned that they used information on quality for performance- based reimbursement programmes to select or reward providers for meeting targets. Finally, information on quality was used for contracting purposes with healthcare providers.

Quality domains

Interpretations of the four quality domains

SAFETY

Safety has been defined as avoiding injuries or harm to patients from care that is intended to help them.13 Healthcare consumers and healthcare providers defined safety as reducing the possibilities of injuries or harm to patients. In addition, the healthcare consumers mentioned an extra aspect, namely:

That the patient feels safe with the healthcare provider. (HC2)

Healthcare providers interpreted safety at an organisational level:

It is important to have an organisation that prevents you from making mistakes or forgetting things. (HP1)

Policy makers and purchasers gave their interpretation at the national level:

Maintain the agreements of safety at a national level. (P2)

EFFECTIVENESS

Effectiveness refers to providing services based on scientific knowledge to all who could benefit (avoiding overuse and underuse).13 All stakeholders interpreted effective care as reaching goals agreed between professionals and patients. When refining the effectiveness domain, however, interviewees mentioned different aspects, such as adherence to ‘evidence-based medicine’, treatment needs to be selected to do least harm or reaching good health outcomes at low costs.

PATIENT-CENTREDNESS

Patient-centredness concerns health care that establishes a partnership among practitioners, patients and their families to ensure that decisions respect patients’ wants, needs and preferences; and that patients have the education and support they need to make decisions and participate in their own care.13 All interviewees mentioned ‘partnership’ in their interpretation of patient-centredness, followed by the degree to which patients’ expectations are met:

Do I offer the patient what he wants, and is he able to utilise it. (HP5)

In addition, the healthcare consumers indicated that each patient should be treated as an individual and should receive care that meets the individual needs.

TIMELINESS

Timeliness has been defined as obtaining care needed and minimising unnecessary delays in getting that care.13 Most respondents defined timeliness as no delays or waiting times before receiving care. From the healthcare consumers’ point of view, ‘a smooth and continuous flow of information through different stages of care’ should be added to this definition. Finally, two providers (HP1, HP4) were not able to define timeliness. Another provider (HP3) considered timeliness to be an inappropriate concept and suggested the use of ‘organisation of care’ instead.

Need for additional domains

All stakeholders added ‘efficiency’ as a separate domain of quality. They described efficiency as avoiding waste and overuse of healthcare services and an acceptable cost per unit of service in the providing of qualitative health care. Furthermore, ‘teamwork’, ‘innovation’ and ‘prevention’ were put forth as separate domains of quality.

Preferences

Patient-centredness was considered to be the most important domain by all stakeholders, except purchasers. This was followed by safety, effectiveness and timeliness. One GP stated:

If care is not patient-centred, you will not achieve your aim whatever you do. (HP6)

Although purchasers underlined the importance of patient-centredness, according to their aim of cost containment, efficiency was rated as the most important quality domain, followed by patient-centredness.

Preferences for structure, process or outcome indicators

Healthcare consumers, providers and purchasers all indicated their preference for process indicators, because these were less sensitive to patient case-mix and to incomplete recording of clinical information. Outcome indicators were considered to have little relation with the quality of provided care and, for that reason, were listed second. Case-mix correction was believed relevant when interpreting results for the public domain and comparing performance. Policy makers, healthcare consumers and healthcare providers all preferred a set of process and outcome indicators. In the long run, purchasers would also prefer process and outcome indicators. Structural indicators were believed to provide basic information without sufficient detail on quality of care provided. One provider stated:

Structure forms the basics, while process is more straightforward. But of course, to fulfil my duties, I have to look at outcomes. (HP1)

In conclusion, process indicators were preferred, and when using advanced data collection and statistical methods, a mix of process and outcome indicators would be preferred.

Discussion

Summary of the main findings

The aims of the stakeholders for using quality indicators varied and overlapped amongstakeholder groups. Stakeholders presented different interpretations of the quality domain safety, primarily due to the scope of their work (which varied from supervising healthcare providers to healthcare providers themselves). When refining definitions, there were differences in definition of effectiveness. In addition, the meaning of the quality domain timeliness was not well understood. Patient-centredness appeared to have a clear, shared focus, but because it was formulated very generally, this agreement was easily achieved. Despite stakeholders’ interpretation of domains varying greatly, their preferences for quality domains appeared to be concordant.

Overall, stakeholders viewed process indicators as most feasible, whereas for the public domain, a mix of process and outcome diabetes indicators was preferred. Some interviewees wanted to include process indicators only, because outcome indicators did not truly assess provider performance if case-mix was not corrected for. These interviewees were not convinced that this shortcoming was clear to all stakeholders.

Strengths and limitations

The number of interviews was small, but we included the most relevant organisations that participate in diabetes quality measurement and improvement, and within these organisations we interviewed those employees whose tasks were most closely related to quality of healthcare assessment or improvement. Even in this group of organisations, with a decisive vote in the selection procedure for national indicator sets, differences in views were substantial. Including more organisations, who would be involved less directly in the indicator selection procedure, would probably have revealed even more variation. Our study design involved a semi-structured interview because we hypothesised that only a discussion could help us answer our questions. In hindsight, an interview lasting one and a half hours is likely to be sufficient considering the schedule of the interviewees, but the discussion of the definition of the quality domains could have easily been longer in most cases.

Aims

The aims of information on quality explored were in line with other studies showing that healthcare providers use quality information for internal quality improvement as well as for the provision of quality information for their healthcare consumers.2,20 Healthcare consumers wanted information on the best providers, waiting times or insurance coverage.21 Furthermore, our results showed that although purchasers felt that they were customer-driven, the domain of patient-centrednesswas not their top priority. Their aims were to increase consumers’ healthcare choices through provision of comparative information on cost and quality and to provide financial incentives to providers for keeping costs low.22–24 Several studies also showed that purchasers and policy makers used quality information to detect areas for improvement, and to implement performance-based reimbursement programmes for healthcare providers. 25,26 Despite differences in the aims of providing information on quality, stakeholders’ preferences for quality domains overlapped to a large extent.

Quality domains

Our study showed that although stakeholders’ interpretations of the domains varied greatly, they had similar preferences for quality domains. Little research has been conducted on this subject, but recently, the Agency for Healthcare Research and Quality (AHRQ) has concluded that the information needs of stakeholders are neither well-refined nor stable.27 The results of our study confirm this. After considering the definitions given by the IOM,13 we conclude that the descriptions given by participants in our study for safety and effectiveness were very general. A proper description of the quality domain effectiveness, which implied evidence-based practice,28 was only mentioned by one provider. According to the IOM, patient-centredness is defined as focusing on the patient’s experience of illness and health care, and on the systems that work or fail to work to meet individual patient’s needs.29 Because of stakeholders’ general answers, their interpretations met this definition; however, we recommend further research on this subject. Delayed care, or timeliness, is a persistent and undesirable feature of current healthcare systems. 29 Timeliness is therefore a legitimate quality domain in health care.30 However, three providers experienced difficulties in interpreting this domain. Furthermore, some interviewees deviated from the IOM definition and this may explain why stakeholders indicated timeliness as being less important.

Needs for additional domains

Cost per unit of service was added as a quality domain by all stakeholders, which could be the result of rising costs, not only in real terms, but also as a percentage of gross domestic product.10 In some definitions of the IOM framework on quality domains, efficiency (including cost) is defined as a separate domain.28 Teamwork, prevention and innovation were other additional quality domains that were suggested by stakeholders. However, teamwork is an organisational aspect, which could be adequately covered by the domains safety and effectiveness.31,32 Prevention and innovation can be considered to be components of effectiveness that are based on the use of systematically acquired evidence to determine whether an intervention, such as a preventive service, produces better outcomes than alternatives. 28 These topics are not sufficiently apparent in current domains of quality and it would be desirable to develop quality indicators that reflect these aspects. The results of our study thus suggest that indicators for general public purposes should reflect the quality domains patient-centredness, safety, effectiveness, timeliness and costs.

Structure, process or outcome indicators

Donabedian’s classic paradigm for assessing quality of care consists of three levels of health care: structure, process and outcome of care. Stakeholders appeared to prefer easily measured process indicators that linked to sound evidence on outcomes. This finding was in line with the results from other studies showing that adjustment to patient case-mix is an important concern for physicians when publishing quality information. 33,34

Conclusions and implications

Indicator selection or development procedures should become more focused on the definitions of quality domains, as well as on the differences in the stakeholders’ perspectives. Different interpretations will lead to different definitions of indicators in the same quality domain. Not addressing these issues could encourage stakeholders to develop new sets of indicators to satisfy their specific aims, creating an additional administrative burden and resulting in a rash of ineffective indicator sets.10 We believe that this problem is not restricted to diabetes care, and probably applies to all quality indicators, as the quality domains in our study are relevant to every health problem. One solution would be to deal with different perspectives during indicator selection or development by taking time to explain definitions and raise different perspectives. This could result in a more efficient development and use of quality indicator sets by stakeholders. In addition, we need to be aware that the relative importance of different quality domains and quality indicators may vary not only between stakeholders, but also over time, which underlines the importance of this exploration as an explicit step in the selection procedure.

Acknowledgements

We thank Ellen Keizer, IQ healthcare, for checking the accuracy of all transcripts against the original recordings.

Funding

The Netherlands Organisation for Health Research and Development (ZonMw).

Ethical Approval

Ethical approval is not necessary for interviews with healthcare professionals in the Netherlands.

Peer Review

Not commissioned; externally peer reviewed.

Conflicts of Interest

None.

References

1. Gagliardi A, Lemieux-Charles L, Brown A, Sullivan T and Goel V. Stakeholder preferences for cancer care performance indicators. International Journal of Health Care Quality Assurance 2008;21(2):175–89.
2. Martirosyan L, Braspenning J, Denig P et al. Prescribing quality indicators of type 2 diabetes mellitus ambulatory care. Quality & Safety in Health Care 2008 Oct; 17(5):318–23.
3. Marshall MN, Romano PS and Davies HT. How do we maximize the impact of the public reporting of quality of care? International Journal of Quality in Health Care 2004 Apr;16(Suppl 1):i57–i63.
4. Fung CH, Lim YW, Mattke S, Damberg C and Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine 2008 Jan 15;148(2): 111–23.
5. Faber M, Bosch M, Wollersheim H, Leatherman S and Grol R. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Medical Care 2009 Jan;47(1):1–8.
6.  Hibbard JH, Harris-Kojetin L, Mullin P, Lubalin J and Garfinkel S. Increasing the impact of health plan report cards by addressing consumers’ concerns. Health Affairs (Millwood) 2000 Sep;19(5):138–43.
7.  Hibbard JH, Stockard J and Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Affairs (Millwood) 2005 Jul;24(4): 1150–60.
8. Marshall MN, Shekelle PG, Leatherman S and Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. Journal of the American Medical Association 2000 Apr 12;283(14): 1866–74.
9. Campbell SM, Braspenning J, Hutchinson A and Marshall MN.Research methods used in developing and applying quality indicators in primary care. BMJ 2003 Apr 12;326(7393):816–19.
10. Braspenning J, Hermens R, Calsbeek H and Grol R. Quality and safety in health care: the role of indicators. In: Grol RWME (ed) Implementation: effective improvement of patient care. Amsterdam: Reed Business, 2011, pp. 195–228 [in Dutch].
11. Ibrahim JE.Performance indicators from all perspectives. International Journal of Quality in Health Care 2001 Dec;13(6):431–2.
12. Klazinga N, Stronks K, Delnoij D and Verhoeff A. Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective. International Journal of Quality in Health Care 2001 Dec;13(6):433–8.
13. Hurtado MP, Swift EK and Corrigan JM.Envisioning the National Health Care Quality Report. Washington, DC: Institute of Medicine, 2001.
14. Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly 1966 Jul;44(3):Suppl 206.
15. Schroeder S, Berkowitz B, Berwick D et al. Performance Measurement.Accelerating improvement. Washington, DC: Institute of Medicine of the National Academies, 2006.
16. McGlynn EA. Identifying, Categorizing and Calculating Health Care Efficiency Measures. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ), 2008.
17. Donabedian A.The Definition of Quality and Approaches to its Assessment. Ann Arbor (MI): Health Administrative Press, 1980.
18. Pope C, Ziebland S and Mays N. Qualitative research in health care.Analysing qualitative data. BMJ 2000 Jan 8;320(7227):114–16.
19. Scientific Software Development. Version Win 6.1. 2009. Berlin.
20. Kocken GA. Medication discussion groups in the Netherlands: five years of experience. Medical Education 1999 May;33(5):390–3.
21.  Harris KM. Choosing a Health Care Provider: the role of quality information. Princeton, NJ: Robert Wood Johnson Foundation, 2008.
22. Schneider EC and Lieberman T. Publicly disclosed information about the quality of health care: response of the US public. Quality in Health Care 2001 Jun; 10(2):96–103.
23. Nakashima E. Doctors rated but can’t get a second opinion: inaccurate data about physicians. Performance can harm reputations. Washinton Post, 2011.
24. Kolstad JT and Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Medical Care Research and Review 2009 Feb;66(1 Suppl):28S–52S.
25. Schut FT and van Doorslaer EK.Towards a reinforced agency role of health insurers in Belgium and The Netherlands. Health Policy 1999 Jul;48(1):47–67.
26. Roland M. Linking physicians’ pay to the quality of care – a major experiment in the United Kingdom. New England Journal of Medicine 2004 Sep 30;351(14):1448– 54.
27. Agency for Healthcare Research and Quality, 2011. wwwtalkingqualityahrqgov/content/create/physician/whats coming aspx 260 J Markhorst, L Martirosyan, H Calsbeek et al
28. Committee on Quality of Health Care in America Institute of Medicine.Crossing the Quality Chasm: a new health system for the 21st century. Washington, DC: National Academies Press, 2001.
29. MurrayMand Berwick DM. Advanced access: reducing waiting and delays in primary care. Journal of the American Medical Association 2003 Feb 26;289(8): 1035–40.
30. Campbell PC, Olufunlayo TF and Onyenwenyi AO.An assessment of client satisfaction with services at a model primary health care centre in Ogun State, Nigeria. Nigerian Quaterly Journal of Hospital Medicine 2010 Jan;20(1):13–18.
31. Baker DP, Amodeo AM, Krokos KJ, Slonim A and Herrera H. Assessing teamwork attitudes in healthcare: development of the TeamSTEPPS teamwork attitudes questionnaire. Quality & Safety in Health Care 2010 Dec;19(6):e49.
32. Kohn LT, Corrigan JM and Donaldson MS. To Err is Human: building a safer health system. Washington, DC: National Academy Press, 1999.
33. Casalino LP, Alexander GC, Jin L and Konetzka RT. General internists’ views on pay-for-performance and public reporting of quality scores: a national survey. Health Affairs (Millwood) 2007 Mar;26(2):492–9.
34. Barr JK, Bernard SL, Sofaer S et al. Physicians’ views on public reporting of hospital quality data. Medical Care Research and Reviews 2008 Dec;65(6):655–73.