Quality in Primary Care Open Access

Keywords

Delivery of health care; Jordan; Counseling; Contraception; Family planning services; Reproduction.

Background

Quality of Family Planning (FP) services support the organized family planning efforts to meet demands of fertility regulation [1,2]. Many programs have applied activities to improve quality of care in their facilities and invested resources to develop appropriate measurements of quality [1-4]. In FP, six elements of quality were suggested by the Bruce/Jain framework: choice of method, information to the client, technical competence, interpersonal relations, mechanisms to encourage continuity, and constellation of services [1,3,4]. However, there is no single comprehensive method of evaluation that implemented all elements of quality. Yet, the most commonly used approaches are direct observation and exit interviews. Direct observation measures the accuracy and thoroughness of information conveyed during counseling and assesses the technical competence of the provider. By contrast, the exit interview is the only instrument that taps the clients view of the services received. Nonetheless, direct observation may instigate a bias among providers performing better than their regular conditions [1-7]. Exit interviews as well may introduce courtesy bias; women may perceive that they are required to give positive feedback on the services, or they can be reluctant to undermine the services for fear of being neglected from care if providers know about their opinion. Eventually, women may simply give any answer without thinking, because they want to get home [1].

Quality of Family Planning (FP) services has long been recognized as the principal determinant of uptake and continuation of contraceptive methods [8-10]. In Jordan, the unmet need for contraception among women who wish not to have any more children have reached 16%, with current contraceptive users among married women of 61%, and total fertility rate at 3.38 [11]. Although FP services are widely known and provided for free in the premises of Jordan’s Ministry of Health (MOH), the information about the quality of these services is limited. In addition, most studies addressed FP in terms of perceptions of providers and clients [12,13], counseling couples on FP [14], practices and attitudes of providers [15,16], and few if none have involved comparing methods of evaluation. Thus, the objective of this study was to investigate the FP quality by comparing two methods of evaluation that are commonly used: the direct observation of health personnel while providing the service, and the exit-interview of women who received FP.

Methods

Design

This was a cross-sectional study. The outcome was the level of agreement between two methods of evaluating the quality of FP services.

Sample and settings

The study took place in three health care centers owned and operated by the Jordanian MOH in Salt city, located in the middle of Jordan. The target population was all married women at reproductive age; between 15 and 49 years old, who attended the designated health care centers for FP services. The initial sample of women approached was 575, but only 503 (87%) agreed to participate in this study. Family planning services at the MOH health centers are provided as part of the daily regular services. Women present at a room where multiple personnel including nurses, midwifes, and nurse assistants provide different tasks such as registering, measuring vital signs, asking and or interacting with women about FP choices, and providing contraceptives. As well, counseling is performed in the same room. The service provided was based on “first come first served”.

Ethical approvals

The present study was approved by the Institutional Review Board (IRB) of Jordan University of Science and Technology (JUST), and the Jordanian MOH. A verbal consent was obtained from the counseling providers before monitoring their work in the counseling room. Participating women were asked to sign a written consent, which included information about the study purpose and benefits, and an affirmation of the voluntary nature of participation in the study. The participants were assured confidentiality, and their names were not recorded.

Data collection

Data was collected from June to July 2013. The data collection for evaluation of FP services was achieved by two methods. First, the researcher witnessed, observed, and recorded the performance of each session where the health personnel provided the FP service. This was done by using a predetermined protocol that included eight items related to performance of the provider. Using the same items, the women who finished and exited the health care center were interviewed and asked to evaluate the service that they just had, and the researcher recorded their responses.

Analysis

Socio-demographic information was obtained using descriptive statistics. The relationship of women opinion about the FP service with: number of children, level of education, and provider years of experience was examined using Chi-Square test. Levels of agreement between observation data and exit interviews data for each client were obtained by Kappa coefficients. Analysis of the data was performed using the Statistical Package for the Social Sciences (SPSS), Version 20 (IBM Corp., Armonk, New York, USA). In all analyses, a p-value of 0.05 was considered statistically significant.

Results

There were 503 sessions of FP services, in which 503 women participated. The mean age of the women was 31.11 years (+ 7.09), about 87% of them were between 20 and 39 years old, and most of them had higher than high school of education (Table 1). Approximately, 41% of women were getting FP services for the first time, 68.6% have had less than three pregnancies, and about 70% wanted to have more children. Among those who were previously using contraceptives, 33.9% used pills, 31.8% used IUD, and 25% used condoms. Regarding the FP providers, 71% had 2 to 4 years college education, and 97% had 4 or more years of experience. The FP service sessions took an average time of 10 minutes, while the counseling lasted for an average of 3 minutes (the women who received counseling comprised 39.4%).

Table 1: Women’s characteristics.

Table 2 shows womens opinions in relationship to education and number of children. Provider’s explanation of how to use the contraceptive method and side effects were significantly related to women’s number of children (p-value=0.024 and 0.004, respectively).

Table 2: Womens opinions of counseling by level of education and number of children.

All other women’s perceptions were not related to education or number of children.

Table 3 demonstrates the observed counseling in relation to providers level of experience, which revealed a significant relationship of almost all items.

Table 3: Observed counseling by providers level of experience.

Table 4 illustrates levels of disagreements the two methods of evaluation of FP services. There was a substantial level of disagreement between the two methods in all items except the item that tackled provider'sexplanation of IUD insertion steps (p-value = 0.008).

Table 4: Percent agreement and kappa values of direct observation and women’s exit interview.

All other women’s perceptions were not related to education or number of children.

Discussion

Quality of care assessments have traditionally centered on medical records. Later, literature has focused on comparing record reviews and exit interviews with direct observation; these studies proved medical chart audit limitations and indicated that methods other than observation can report consistently on some, but not all aspects of health care process [2,17]. Exit interviews data can measure aspects of quality that are more subjective, such as respectful treatment, client satisfaction, methods discussed by the provider, waiting times, and information given during the counseling session on topics including side effects, method use, and when to return to the facility [1,4,7,9]. Learning more about these processes with the aim of improving them can have important programmatic payoff [8,10,18]. In this study, we restricted the scope of questions to the relevant objectives, so that our questioning would not represent a time constrain for women.

In general, the results of this study showed high disagreement between the data obtained from direct observation and exitinterviews. The women recipients of FP care highly appraised performance of providers during the service sessions (results ranged from 97% to 99.6%, except for one item; 32.8%). On the contrary, researchers evaluated the service in much lower rates, ranging from 2% to 52%, except for one item; 85%. These findings may indicate that the women had high appreciation of the providers performance regardless of the status of care they had, which was shown in almost total satisfaction with providers care. Such perception may be justified while considering that womens priority was not to criticize health care providers, especially that they needed to return for follow-up visits. Similar conclusions were reported by other studies about women’s perceptions of health care facilities and services [7,19,20]. Further, the reliability of exit interview data relies on womens memory, level of attention toward the provider’s actions, expectations about the level of care by the provider, and comfort when talking to an interviewer [7,19].

While women in this study were generally satisfied, other studies reported women’ satisfaction with specific items during counseling [8,12,19,21]. However, only 39.4% of women in this study have received FP counseling, a service that somewhat could be described as inadequate (average time was 3 minutes), and the rest of women did not receive any form of counseling; they were provided with contraceptives only. This may indicate that women in this study were not aware of qualities of an adequate FP services, otherwise, they would have relatively reflected that in their views of the service [7,19]. Although the analysis in this study did not include all women’ characteristics, other factors such as ever use of FP services, desire for more children, and the type of previously used contraceptive methods may have played a role in forming perceptions about providers’ performances. These factors were not analyzed due to the negligent portion of women who described the performance as inadequate. Overall, women’ perceptions of the FP services cannot be solely used as a method of evaluation because disagreement with direct observation was high. In addition, other studies indicated that women tend not to criticize health settings that are familiar to them, fearing that criticism would be interpreted as offensive [7,13].

In our study, the researchers performed an objective-based evaluation which entitled checking for existing standards of care and avoiding personal perceptions. Direct observation intends to measure gaps between desired and actual quality of care delivered [8]. It’s not conservative or strict measurement of quality, but rather an objective evaluation. Providing adequate FP services were affected by providers experience, as indicated by the statistically significant relationship of almost all items observed. Moreover, the adequacy of the provided services has increased proportionally in relation to years of experience. In this aspect, high percentages of studies of contraceptive discontinuation rates have indicated that the principal reason for discontinuation is dissatisfaction with the quality of service [17,21]. Although our results showed unexplained high womens satisfaction with the offered FP services, Jordan’s discontinuation rate and unmet need for FP are still at high rates; 11% and 16%, respectively [11]. It is imperative for the Jordan’s health care system to put forward strategies and policies for continuous monitoring of health care services, in addition to regular evaluation measures.

Our study introduced better understanding of the comparison between direct observation and exit-interview. Many studies attributed the disagreement between the results obtained from applying the two methods to multiple factors, including: courtesy bias, reliability of reporting, the Hawthorne effect, and recall bias were discussed [4,7,13,22-24]. When clients do not report negative aspects of care, or feel uncomfortable doing so, this might be due a courtesy bias [23]. Provider interviews may be unreliable due to providers intentions to show an ideal performance of service delivery rather than what they do in practice [23]. As well, the Hawthorne effect which results when providers demonstrate their best behavior during observations [23]. Most of these information biases may skew the resulting measures of quality in a positive direction of higher perceived quality [4,7,13,23]. In these cases, biased information may result in false study implications and conclusions [23].

With the continuous debates on drawbacks in the provision of health care services, adequate evaluation is a golden standard strategy for quality assurance [17,18,23-26]. Many studies recommend approaches that apply multiple methods of evaluation [4,7,23]. In addition, the method itself should be carefully designed to explore all possible dimensions of evaluation of the quality of health care services [7].

Conclusion

The evaluation of quality of FP services reported in this study has spotted important findings that concerns health care providers and researchers. Client satisfaction interviews should not be used without direct observation, as a quality measurement, in settings where health care providers and clients are well-known to each other. Recommendations for further research includes conducting a longitudinal study on a national scale that follow up with women who attend certain health care centers for family planning services to evaluate the quality of services at different time points and examine any variations noted in findings of these evaluations. This will help to improve the quality of the evaluation process and work on enhancing services based on these evaluations.

Limitations

One major limitation is the cross - sectional nature of the study that only captured womens perception at a certain point and not necessary reflect their opinion over a longer period; for instance, their opinion from all visits since they started visiting a certain family planning health service center. Furthermore, sample population were limited to a small geographical region that certainly affected the generalizability of the findings. Another possible limitation is that womens responses in the exit interview might have been impacted by their expectations of that visit, date scheduled for the next visit, or womens impressions about the service from previous visits, which could have affected their overall satisfaction, and thus, altered their responses.

Funding

There is no funding received for this study.

Acknowledgment

We would like to thank Jordan University of Science and Technology and Jordan Ministry of Health for their support and agreements, and women who agreed to participate in the study.

References

1. Bessinger R, Bertrand J. Monitoring Quality of Care in Family Planning Programs: A Comparison of Observations and Client Exit Interviews. Int Fam Plan Perspect 2001. 27: 63-70.
2. Tumlinson K. Measuring Quality of Care: A Review of Previously Used Methodologies and Indicators, in Working Paper Two of the Measuring and Monitoring Quality of Services and Quality of Care Project. 2016: New York: Population Council.
3. Bruce J. Fundamental elements of the quality of care: a simple framework. Stud Fam Plann 1990. 21: 61-91.
4. Donabedian A. The quality of care. How can it be assessed? JAMA 1988. 260: 1743-1748.
5. Franco LM, Franco C, Kumwenda N, Nkhoma W. Methods for assessing quality of provider performance in developing countries. Int J Qual Health Care 2002. 14: 17-24.
6. Department of Statistics and ICF International, Jordan Population and Family Health Survey 2012. Calverton, Maryland, USA: Department of Statistics and ICF International 2013.
7. Jain AK. Fertility reduction and the quality of family planning services. Stud Fam Plann, 1989. 20: 1-16.
8. Assaf S, Wang W, Mallick L. Quality of care in family planning services in Senegal and their outcomes. BMC Health Serv Res 2017. 17: 346.
9. Hutchinson PL, Do M, Agha S, Measuring client satisfaction and the quality of family planning services: a comparative analysis of public and private health facilities in Tanzania, Kenya and Ghana. BMC Health Serv Res 2011. 11: 203.
10. Mpunga D, Lumbayi JP, Dikamba N, Mwembo A, Ali Mapatano M, et al. Availability and Quality of Family Planning Services in the Democratic Republic of the Congo: High Potential for Improvement. Glob Health Sci Pract 2017. 5: 274-285.
11. Al-Qutob R, Nasir LR. Provider perceptions of reproductive health service quality in jordanian public community health centers. Health Care Women Int 2008. 29: 539-50.
12. El-Khoury M, Thornton R, Chatterji M, Kamhawi S, Sloane P, et al. Counseling Women and Couples on Family Planning: A Randomized Study in Jordan. Stud Fam Plann 2016. 47: 222-238.
13. Kamhawi S, Underwood C, Murad H, Jabre B. Client-centered counseling improves client satisfaction with family planning visits: evidence from Irbid, Jordan. Glob Health Sci Pract 2013. 1: 180-92.
14. https://share-net-jordan.org.jo/en/content/jordan-private-doctors-family-planning-and-breast-cancer-survey
15. https://share-net-jordan.org.jo/sites/default/files/Hormonal_1.pdf
16. RamaRao S, Lacuesta M, Costello M, Pangolibay B, Jones H, et al. The link between quality of care and contraceptive use. Int Fam Plan Perspect 2003. 29: 76-83.
17. Akachi Y, Kruk ME. Quality of care: measuring a neglected driver of improved health, Bulletin of the World Health Organization. 2017.
18. Tessema GA, Streak Gomersall J, Mahmood MA, Laurence CO. Factors Determining Quality of Care in Family Planning Services in Africa: A Systematic Review of Mixed Evidence. PLoS One 2016. 11: 1-23.
19. Waqas Hameed, Syed Khurram Azmat, Moazzam Ali, Wajahat Hussain, Ghulam Mustafa, et al. Determinants of Method Switching among Social Franchise Clients Who Discontinued the Use of Intrauterine Contraceptive Device. Int J Reprod Med 2015. 15: 941708.
20. Williams JR, Gavin LE, Carter MW, Glass E. Client and Provider Perspectives on Quality of Care: A Systematic Review. Am J Prev Med 2015. 49: 93-106.
21. Mohammad-Alizadeh S, Wahlstrom R, Vahidi R, Johansson A. Women's perceptions of quality of family planning services in Tabriz, Iran. Reprod Health Matters 2009. 17: 171-80.
22. Nasr E, Hassan H. Association between quality of family planning services and client’s satisfaction level in maternal and child health centers in Port Said city. J Nurs Educ Pract 2016. 6: 85–99.
23. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol 2014. 67: 267-277.
24. Al-Abri R, Al-Balushi A. Patient satisfaction survey as a tool towards quality improvement. Oman Med J 2014. 29: 3-7.
25. Alrashdi I. Evaluation of quality of healthcare: to what extent can we rely on patient expectations and preferences. Oman Med J 2012. 27: 448-449.
26. Tumlinson K, Speizer IS, Curtis SL, Pence BW, et al. Accuracy of standard measures of family planning service quality: findings from the simulated client method. Stud Fam Plann 2014. 45: 443-470.