Research Paper - (2011) Volume 19, Issue 3
Associate Professor of Pediatrics, Director, South Carolina Pediatric Practice Research Network, Medical University of South Carolina, Charleston, SC, USA
Katherine D Freeland BS
Research Specialist, Department of Pediatrics, Medical University of South Carolina, Charleston, SC, USA
Maureen S Kolasa RN MPH
Child Team Lead, Health Services Research and Evaluation Branch, Immunisation Division, National Center for Immunisation and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, USA
James T McElligott MD MS
Assistant Professor of Pediatrics, Medical University of South Carolina, Charleston, SC, USA
Paul M Darden MD
Professor of Pediatrics, Section Chief, General and Community Pediatrics, Oklahoma University Health Sciences Center, Oklahoma City, OK, USA
Received date: 29 October 2010; Accepted date: 15 March 2011
ObjectiveImmunisation coverage of children b19 months of age in US primary care practices is below the desired goal of 80%. In order to improve this rate, primary care providers must first understand the specific processes of immunisation delivery within their office settings. This paper aims to identify key components in identifying strategies for quality improvement (QI) of immunisation delivery. MethodsWe surveyed a South Carolina Pediatric Practice Research Network (SCPPRN) representative for each of six paediatric practices. The surveys included questions regarding immunisation assessment, medical record keeping, opportunities for immunisation administration and prompting. Subsequently, research staff visited the participating practices to directly observe their immunisation delivery process and review patient charts in order to validate survey responses and identify areas for QI. ResultsMost survey responses were verified using direct observation of actual practice or chart review. However, observation of actual practice and chart review identified key areas for improvement of immunisation delivery. Although four practices responded that they prompted for needed immunisations at sick visits, only one did so. We also noted considerable variation among and within practices in terms of immunising with all indicated vaccines during sick visits. In addition, most practices had multiple immunisation forms and all administered immunisations were not always recorded on all forms, making it difficult to determine a child’s immunisation status. ConclusionsFor any QI procedure, including immunization delivery, providers must first understand how the process within their practice actually occurs. Direct observation of immunisation processes and medical record review enhances survey responses in identifying areas for improvement. This study identified several opportunities that practices can use to improve immunisation delivery, particularly maintaining accurate and easy-to-locate immunisation records and prompting for needed immunisations during sick visits.
immunisation procedures, immunis-ations, once-based preventive services, quality im-provement
Immunisations are perhaps the most cost-effective medical interventions to enhance the wellbeing of children.[1,2] However, achieving adequate immunisation coverage of children in the USA is often a challenge. Statistics from the National Immunisation Survey (NIS) conducted by the Centers for Disease Control and Prevention (CDC) show that in 2008 the national immunisation rate for the 4:3:1:3:3:1 series of vaccines (four diphtheria, tetanus, and acellular pertussis vaccines (DTaP), three polio vaccines (IPV), one measles– mumps–rubella vaccine (MMR), three Haemophilus influenza type b vaccines (Hib), three hepatitis B vaccines (Hep B), and one varicella vaccine (Var) in children aged from 19 to 35 months) was 76.1%, which is below the Healthy People 2010 objective of 80%.[3,4] Furthermore, in 2008 only 61.2% of children received the 4:3:1:3:3:1 series by the age of 19 months, when all vaccines from this series were actually due or overdue, indicating that these children did not receive some of their vaccinations at the recommended age.[4] Other studies have also demonstrated that only 9% of children receive every individual childhood vacci-nation at the recommended age.[5–7] These statistics suggest that the majority of children in the USA are under-vaccinated at ages at which they are most vulnerable.[6]
Series completion rates tend to be driven by the vaccine doses given in the second year of life, with the primary laggard being the fourth dose of DTaP.[8] The 2008 NIS data show that while 96.2% of children aged between 19 and 35 months were given three DTaP vaccinations, only 84.6% of children received the fourth DTaP.[4] The 2008 NIS data report that only 69.4% of children received the fourth DTaP before the age of 19 months.[4] Dombkowski et al reported that some simulation models predict that children who receive the complete four-dose DTaP series more than six months late are, in the course of one year, up to 42% more likely to get pertussis than children who receive the series on time.[9] Therefore, improving the delivery of the fourth DTaP is important in strength-ening the child’s immunity to disease and will assist in increasing the national immunisation completion rates.
Most children in the USA, including those living in poverty, receive their immunisations in the primary care once setting, usually paediatrics or family medi-cine.[10–12] The Task Force on Community Preventive services (TFCPS) gives several recommendations to improve childhood immunisation coverage in once settings. These recommendations include patient reminder/recall, educational interventions for both patients and staff, community based and clinic-based interventions, incentives for patients and families, a list of vaccination requirements for school attendance and patient-held immunisations.[13] A similar list of recommendations was published in the Standards for Child and Adolescent Immunisation Practices.[14]
Preventive services, including immunisations, are often poorly delivered and immunisation rates vary widely between practices.[11,15] A challenge to imple-menting immunisation recommendations is the vari-ation of delivery of preventive care services in each individual practice.[16,17] For example, once practices vary widely in their delivery of immunisations at sick and well visits, immunisation documentation, pro-vider education and feedback regarding immunis-ation rates, vaccine ordering, patient reminders and recall systems. Little information has been published on the extent to which these recommendations are implemented. Parent refusal and parental choice to adopt alternative immunisation schedules also have an effect on immunisation rates, though ways to intervene in this case are beyond the scope of this paper.[18,19]
When planning any once based effort, whether it is to improve immunisation rates or change another aspect of the practice, a key component is to first understand the current process of vaccine delivery in the once. This paper aims to identify key components in identifying strategies to initiate quality improve-ment (QI) procedures for immunisation delivery within the practice setting.
This study is part of a larger QI project aimed at improving the delivery of childhood vaccines in the practice setting. In order to develop interventions for each of the practices, it was necessary to first identify the process by which each practice delivers immunis-ations. Variables include factors such as record keeping, types of visits during which vaccines are administered, once systems that determine when children are due for immunisations, and the ability and decisions to vaccinate at sick visits as well as at health maintenance visits. These variables are unique to each practice and must be assessed in order to identify appropriate areas for improvement. Following the identification of im-munisation processes for each practice, an inter-vention was developed by each practice to improve their vaccine delivery. This study was reviewed and approved by the Institutional Review Board of the Medical University of South Carolina (MUSC).
The study took place in the South Carolina Pedi-atric Practice Research Network (SCPPRN), a regional network of paediatric practices established in 2005. At the time the study was initiated in 2007, there were nine practices in the SCPPRN, with 26 paediatric providers. The six practices that chose to partici-pate in this study represented approximately 95 375 paediatric patient visits per year with two practices each in urban, suburban and rural locations. Table 1 lists the characteristics and composition of each of the six practices that participated in the QI immunisation project.
A survey inquiring about the paediatric practices’ current immunisation policies, procedures and rou-tines was adapted from the Medical University of South Carolina Teaching Immunisation Delivery Education (TIDE) module. The TIDE module can be found at www2.edserv.musc.edu/tide.[20] The adapted survey was piloted with physicians and nurses, and following revisions based on pilot testing the survey was delivered to each participating SCPPRN practice rep-resentative. The survey, shown in Table 2, consisted of 20 ‘yes/no’ questions and included categories of as-sessment of immunisation coverage, medical record keeping, opportunities for immunisation adminis-tration, provider prompting for needed immunis-ations and reminder/recall of patients. The SCPPRN representative from each of the six practices com-pleted the survey.
The survey was completed as part of preliminary data for a cooperative agreement application. We received the award approximately nine months after the survey data were obtained. In preparation for identifying the QI interventions, we set out to validate the responses to the survey and to identify areas for QI. A research assistant and one of the investigators visited each of the participating practices for the specific purpose of directly observing their immunisation delivery process over a two to three-hour period. The research staff also reviewed the medical records, noting the number of places within the medical record that immunisations were uniquely recorded and the presence of any type of prompting system on the chart or on any computer-generated encounter sheet. Other answers from the survey could be validated from observation in the practice, such as identifying the immunisation schedule and a guide to contraindi-cations posted somewhere in the practice, or by talking with the nursing and front once staff. Differ-ences between survey responses and validation findings were compared using McNemar’s test. The findings were discussed with the practices and were used to develop quality improvement changes in the practices.
Observation of the immunisation delivery process and chart review revealed that survey responses usually reflected actual practice. Of note, all of the practices reported ‘yes’ to the following survey questions: medi-cal records are routinely available at the time of all visits, immunisations are given at all types of visits, an immunisation schedule is posted as a reminder to staff and routine immunisation education is given to patients and parents. All but one practice reported that a guide to immunisation contraindications is posted or readily available. Five practices indicated they used a prompt for immunisations that were due for children pre-senting at well child visits. Validation of the survey responses indicates that these responses did reflect routine practice. The frequency of responses to all 20 questions is listed in Table 2. The percentage of agreement corresponds to the number of practices whose answer (whether ‘yes’ or ‘no’) was the same upon validation. For example, if five practices reported ‘yes’ to prompting for immunisations at well visits, and one practice reported ‘no’, there was 100% agree-ment if the same five practices did use a prompt for a well visit and the sixth practice did not. The paediatric practitioners’ perception of their practice’s use of prompting for immunisations at sick visits did not always match actual process. Four out of six paedi-atricians indicated on the survey that their practice prompted for administration of age-appropriate im-munisations during sick or acute care visits, but chart review and observation showed that only one practice had a verifiable prompt on the medical record for all sick visits. Although the difference was not statistically significant, direct observation and chart review did help some of the practices identify this as an area for QI.
Direct observation and chart review also identified another issue with sick visits in all of the practices. All practices reported that they routinely administer all indicated vaccines at a visit and, in accordance with this, we noted that in all practices some children did receive immunisations during sick visits. However, we also noted considerable variation among and within practices in terms of immunising with all indicated vaccines during sick visits. Some paediatricians im-munised children with a low grade fever (between 100.48 and 101.58 Fahrenheit) while others would not immunise children with any elevation in temperature. No consistent process of immunising a febrile child between or within practices could be identified. Fur-thermore, during sick visits in at least two of the practices, some children who did not have a fever still did not receive needed immunisations.
Another barrier to delivery of immunisation ser-vices was identified through examination of immu-nisation records. Although all six practices reported that ‘immunisations are recorded in a single easy-to-locate place in the chart and/or electronic record’, this was true in only one of the practices (which is not statistically significant). Most practices had multiple immunisation forms and all administered immunis-ations were not always recorded on all forms, making it dincult to determine a child’s immunisation status. In some cases, immunisations were not transcribed to the medical record when patients transferred into a practice and were only found by looking at the transfer paperwork. In one practice, influenza immunisations were generally not recorded with other immunisations but on a single sheet in the progress notes. In two of the practices with electronic medical records (EMRs), immunisations could be viewed in several locations, including a table listing all immunisations and two public health department forms, one of which only included the immunisations required for school entry and an older form that included all immunisations. Even in the one practice that had a single location for recording immunisations, these immunisations were recorded by both generic and trade name and in alphabetical order, making it dincult to determine if patients’ immunisations were up to date.
Two of the survey questions could not be validated based solely on observation. These questions were as follows:
1 providers receive feedback on immunisation rates compared to other providers/practices and
2 immunisation records from previous providers are routinely available at the time a new patient trans-fers into the practice.
While we anticipate that most practices do receive records when a new patient enters the practice, there were insuncient numbers of new patients entering practices during the visits by research staff to enable them to observe enough actual events.
None of the practices had a recall system in place for children who were overdue for immunisations and had not been recently seen. Upon validation, we were able to confirm that there was no recall system in place in any of the practices.
In order to implement a strategy to improve immu-nisation delivery, providers must first understand how the process of immunisation within their practice actually occurs. Survey procedures are often used to gather information about the processes that occur in the once setting.[16,21,22] In our study, practices reported their impression of how immunisation delivery oc-curred in their practice and then we conducted direct observation of immunisation practices and a review of immunisation records within the medical charts. Immunisation processes usually agreed with the pro-viders’ responses in the survey. However, observation of actual practice provided key information which would have been missed through use of a survey alone. In addition, sometimes provider perception and actual process differed. Had we relied solely on the survey without verification, we would likely have developed unnecessary interventions or ignored potentially help-ful ones. The survey and subsequent validation within each practice allowed the practices and the research team to understand the difference between perception and practice. This understanding fostered dialogue which led to more options for improving the immu-nisation process at the practice level. When possible, effort should be made to confirm physician survey data by a combination of observation, medical record review and outcome measurement.
Observation of immunisation practices also iden-tified processes that hindered encient delivery of vaccines. For example, we observed striking differ-ences in record keeping among practices and issues in record keeping in all six practices. In five of the six practices, immunisation records were found in mul-tiple locations with varying levels of completeness, resulting in dinculty in assessing a child’s immunis-ation status. Recording immunisations in a consistent manner rather than using multiple names (i.e. anti-gens, brand names) would also simplify review of a child’s up-to-date status. Information about the vac-cines the child has already received is fundamental to improving vaccine delivery. Observation of practice processes allowed for identification of issues with immunisation record keeping in all practices.
Along with highlighting the need to maintain im-munisation records in one easy-to-locate place in the medical record, this study helped identify several other opportunities that practices can use to improve immunisation delivery. Some type of prompt, whether physical or electronic, depending on medical record type, should be implemented to serve as a reminder to immunise during sick visits. In addition, standing orders for adult immunisations have been strongly recommended by the TFCPS as well as by the CDC, specifically for adult influenza vaccination.[13,23] The studies supporting these recommendations were con-ducted in the adult inpatient setting.[24–26] However, given the success of these interventions, it is reason-able to consider that this strategy may also be useful in improving coverage for children.
Finally, a recall system for patients who fail to keep appointments should be considered, using strategies that promote enciency in recall systems, such as computer-based systems or tickler card systems. Another example may be a recall system that tracks patients who did not come in for their appointment at the end of each day and links this to an automatic telephone dialling system to send a standard recorded phone message. All components should be part of an over-arching QI initiative, as this has been demonstrated to be more effective than prompting alone.[12,27–29]
The above potential improvement processes are recommended by the TFCPS.[13] Practice-based QI and systems change have been used with great success to improve the delivery of preventive services.[27,28] One study examining a provider prompting inter-vention in a randomised control trial found that prompting residents for the immunisations that were currently due significantly improved immunisation delivery by reducing missed opportunities (11.4% vs 21.6%; P <0.001).[30] While the above intervention was used for well child visits, and our data suggest that most practices already prompt at well child visits, this prompting intervention would probably be effective for the sick visit.
This study is limited to a small convenience sample of six practices in eastern South Carolina and the results may not be generaliseable to other practices or regions. We were unable to validate all survey re-sponses, and interpretation of survey questions may have varied. We also observed the process at one point in time, rather than conducting numerous visits over a longer period of time. Therefore, we could not verify that some of the procedures we observed occur all of the time. In addition, we did not use a specific procedure of having two observers record their ob-servations with a measure of inter-rater reliability. This could have significantly improved the validation process, however, these practices probably have simi-larities to many other paediatric practices, and the methods used can be adapted for use by other practices striving to improve immunisation service delivery.
For any quality improvement procedure, including immunisation delivery, providers must first under-stand how the process within their practice actually occurs. Direct observation of immunisation processes and medical record review enhances survey responses in identifying areas for quality improvement. This study helped identify several opportunities that prac-tices can use to improve immunisation delivery, par-ticularly maintaining accurate and easy-to-locate immunisation records and giving prompts for needed immunisations.
This project was presented in part at the 2009 National Immunisation Conference, Dallas, TX, USA and at the 2009 Pediatric Academic Societies Meeting, Baltimore, MD, USA. The authors would like to thank the fol-lowing South Carolina Pediatric Practice Research Network practices for their participation: Barnwell Pediatrics, Barnwell, SC, Charleston Pediatrics, Charleston, SC, Georgetown Pediatrics, Georgetown, SC, MUSC Children’s Care, N. Charleston, SC, MUSC Pediatric Primary Care, Charleston, SC, Seacoast Pediatrics, Charleston, SC.
This project was supported by grant 5U01IP000146 from the Centers for Disease Control and Prevention. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Preven-tion, US Department of Health and Human Services.
This study was reviewed and approved by the Insti-tutional Review Board of the Medical University of South Carolina (MUSC).
Not commissioned; externally peer reviewed.
None.
