Trauma & Acute Care Open Access

Keywords

Social integration; Disability evaluation; Veteran; Caregiver; Participation

Introduction

Community reintegration is a primary intervention goal for veterans with disabling physical conditions and mental disorders; and thus a priority area for outcomes assessment in the Department of Veterans Affairs (VA). The need for measurement of veteran community integration has been highlighted by reports of challenges in post-deployment reintegration for veterans deployed in Operation Enduring Freedom (OEF), Operation Iraqi Freedom (OIF), and Operation New Dawn (OND), military campaigns collectively classified under the umbrella of “the Global War on Terror” (GWOT). A national survey of OEF/ OIF veterans found that 25% reported difficulties in major life domains [1]. Amongst GWOT Veterans seeking care at VA facilities 40% experience a variety of community integration problems, such as difficulties with relationships, struggling to hold a job, and increased isolation [1,2]. Roughly 96% were interested in treatment for community integration issues [2]. About 50% of OEF/OIF Veterans participating in a large clinical trial reported reintegration difficulties [1].

Traumatic injuries are frequently accompanied by mental health disorders, such as posttraumatic stress disorder (PTSD), depression, and anxiety [3]. Many more service members, not classified as “wounded in action”, have been diagnosed with traumatic brain injury (TBI) and/or PTSD. An estimated 15- 19% of OEF/OIF/OND service members sustained a TBI during deployment [4]. As of May 7, 2014, TBI was diagnosed in over 300,000 U.S. service members [2]. TBI/Polytrauma is associated with cognitive, psychosocial, and behavioral problems, including memory deficits, attention difficulties, and irritability. Amongst OEF/OIF Veterans, 89% with a TBI diagnosis had a comorbid mental health diagnosis; 73% of those diagnoses were PTSD. Prevalence estimates of post-deployment PTSD range from 5-20% of OEF/OIF Veterans, translating to 100,000-400,000 recent combat Veterans with PTSD. PTSD is associated with diminished quality of life and multiple problems in community integration such as aggressive behavior, domestic violence, ineffective parenting, unsafe driving, unemployment, and social alienation [5,6].

Although community reintegration care is an integral goal of VA care and is emphasized in areas of physical medicine and rehabilitation, mental health, and primary care, there is no system-wide, standardized approach to its measurement. Yet, measurement is essential for developing and testing interventions targeting enhanced participation, documenting clinical program effectiveness and tracking population health. In 2010, the VA Working Group on Community Integration, agreed that the construct of participation as defined by the WHO’s International Classification of Health and Function (ICF) [7] was an appropriate conceptual framework for defining and measuring community integration [8].

Considerable advances have been made in the measurement of veteran community integration using patient self-report measures (PROMS) developed based on the ICF framework. The Community Reintegration Measure for Injured Service Members (CRIS) [7] and the computer adaptive test version, the CRIS-CAT [9], were developed to assess issues specific to injured service members. The CRIS demonstrated good reliability, construct and predictive validity in samples of Iraq and Afghanistan warera veterans [10]. PROMS, by definition, represent the patient's perspective, a necessary and important metric of community integration. However, in clinical situations where illness or disability prevents completion of PROMS, or the patient has limited insight, or symptoms or stigma distort self-report, reliance on self-report alone is not possible or advisable. Yet, excluding data from these types of patients would bias results of research studies, and leave clinicians without important data. Thus, an alternative or additional measurement approach to community reintegration measurement using proxies is needed.

Furthermore, authorities strongly recommend that in assessment of patients with PTSD or TBI information obtained via patient self-report and/or, clinician rating should be supplemented, whenever possible, with collateral data from friends, family members, coworkers, or supervisors [6,11]. Given that health care utilization is influenced by proxy perceptions and clinicians routinely consider data from multiple sources in making care decisions, proxy ratings may provide an important source of information in many other situations. Thus, the objectives of this study were to 1) create and pilot test a proxy version the CRIS, which we called the CRIS/P; 2) create and pilot test a measure of proxy satisfaction with Veteran community integration.

Impact and Implications

Although community reintegration is an important rehabilitation goal, standardized approaches to its measurement are limited.

Two new proxy measures related to Veteran community integration were developed and validated.

Proxy ratings used by clinicians may provide important information for clinical decision-making.

Methods

Conceptual framework

In this study, we utilized the measurement tool that was developed using the proxy-proxy perspective as defined by Pickard and Knight in 2005 to develop the CRIS/P [12]. Pickard and Knight introduced a framework for proxy measures that included two types of measures which they called 1) a “proxypatient” measure in which the proxy rates responses according to how he/she thinks that the patient would respond; and 2) a “proxy-proxy” measure in which the proxy rates responses from their own perspective.

Historically, few measures have been developed using a clear framework and specific set of instructions. Instead, most proxy measures use the same measurement instrument for both patient and proxy with no explicit instruction given regarding the perspective from which the proxy is to complete the measure. In some cases, proxy measures have altered wording so that the questions are framed in the third person when asking the proxy, but they do not provide any explicit instruction regarding the proxy perspective. However, there is ample evidence that the type of perspective that the proxy takes influences their ratings of disability and quality of life, and thus is associated with the inter-rater gap between proxy and patient perspective. Pickard’s study of Veterans with cancer found statistically significant differences in scores on the EQ-5D and Cancer Quality of Life Questionnaire between a proxy-proxy and proxy-patient version with the proxy-patient version more closely correlated with the patient self-report [13]. Lobchuk compared 3 types of proxy measures (neutral, proxy-patient and proxy-proxy perspectives) in a study of cancer patients and reported that the proxy-patient responses were more strongly correlated with patient responses [14]. McPhail’s study of proxy-patient agreement on the Euroqol- 5D in geriatric rehabilitation patients found strong agreement between proxy-patient assessments and patient self-report at discharge across all cognition levels.

The CRIS measure

The CRIS measure has three sub-scales, each measuring a different dimension of participation. Each of the sub-scales is comprised of items drawn from the 9 activity and participation content domains (or chapters), defined by the ICF. The Perceived Limitations to Participation subscale assesses veteran’s perceived limitations in participation. The Extent of Participation subscale assesses how often veterans experience a challenge in participation. The Satisfaction with Participation subscale assesses veterans’ level of satisfaction with participation. The instrument measures objective and subjective elements of participation and includes items related to negative as well as positive aspects of participation [15]. Prior research showed that the scales had good reliability, a broad spectrum of item difficulty, and evidence of concurrent and construct validity [10,15].

Study design

The study had two phases. The first phase was cognitive testing of the newly developed CRIS/P and Proxy Satisfaction measures. The second phase was a reliability study of the final measures.

Sample

Participants in both phases were caregivers of veterans recruited through local VA health professionals, advertisements, pressreleases and online posting sites. Eligible participants met the following inclusion criteria: ages 18-80, spouse, partner or loved one (parent, sibling or other caregiver) of a veteran with mental or physical health problem(s) that were treated within the DoD and/or VA health care systems for mental and/or physical health problem(s), able to understand the requirements of the study and provide informed consent. Phase II participants were excluded if they were unable to commit to attending two visits within a one week period.

Phase I: Cognitive Testing

Data collection

Prior to data collection all CRIS items and instructions were revised into proxy-proxy formats by members of the research team. Additionally, the CRIS satisfaction scale was revised to address caregivers own satisfaction with elements of their veteran’s participation. The content of the proxy items was designed to mirror that of the original items, but differ by including proxy specific instructions. Several alternatives for item stems, response categories and wording were generated and discussed by the study team and a plan was developed to test the alternatives in cognitive interviewing.

Participants took part in a single session of about 1.5 h in which they reviewed and discussed the proxy items. Each cognitive testing session covered between 10-20 items in each of the three CRIS-scales. Participants were instructed to answer the items from their own perspective and then were asked to answer the same item using the alternative different wording and response formats. After completing the survey items participants were probed on the wording and format. Respondents were asked to talk about their response process including their comprehension of the item, their ability to recall the answer, and their strategy of retrieving information related to the question. Specific probes asked questions such as: Do the questions mean the same thing to you? If not, why not? Is one clearer/easier for you to understand? If so, which one? What makes it clearer (easier) for you? What were you thinking or feeling when you heard it that made the second way less clear? Was there a format of the question that you preferred? Was there a particular format of the question that you disliked or found confusing? Which one and why?

Cognitive testing data analysis and item revision

All cognitive interviews were audiotaped. Two analysts (the original interviewer) and a second qualitative analyst listened to the audiotapes to extract responses to the above items. The proportion of participants who indicated a preference for each type of response was tallied. Suggestions for additional item revisions based on participants’ responses were also documented. These findings were discussed with the Principal Investigator and decisions on the best wording and format of items were made. Data analysis, item revision and data collection was an iterative process. The results of each cognitive testing session were debriefed and alternative wording of items or response generated. These refinements in the wording of items were then cognitively tested in subsequent interviews. The final result of the cognitive testing phase was a refined preliminary CRIS/P survey that was then used in Phase II.

Results

Phase I: Cognitive testing

Cognitive testing was conducted with 10 participants who were caregivers of male veterans (Table 1). 30% of veterans were from the Vietnam era, 20% from the Gulf War, 20% from OEF/OIF, and 10% from other eras. All caregiver participants were female, 90% were white, 10% black, 90% non-Hispanic, 10% Hispanic. 90% were spouses and lived with the veteran. All saw their Veteran on a daily basis. Refinements to the CRIS-Proxy item set were made resulting in the version used in Phase II testing.

Table 1: Demographics of participants in each of the samples.

Phase II: Pilot Testing

Data collection

The preliminary CRIS/P and the Proxy Satisfaction measure were administered to 24 caregivers. Twenty three completed the measure on two occasions within 7 days. Basic information regarding the proxy and Veteran’s age, gender, race, relationship, employment status and medical and mental health history was also collected.

Data analysis

The characteristics of participants and scores of the CRIS/P and Proxy Satisfaction items and subscales were examined descriptively. Responses to items pertaining to employment and parenting were missing for the majority of respondents because these questions were not applicable to their Veteran. Therefore, these sparsely populated items were removed to enable scale analyses. The item to test, item to total and scale alphas of each of CRIS/P subscale at the first test administration was examined. After inspection of results, items with correlations below 0.1 were eliminated. Several items with low item-total correlations (0.1-0.2) were retained because of their conceptual importance. The internal consistency analyses were then rerun with the truncated item set.

Test re-test reliability of individual items was examined for all items that had a minimum of 7 respondents using the Shrout and Fleiss intra-class correlation coefficient (type 3,1), a two-way mixed model, single measure of reliability. Items with missing data, but considered conceptually important were then added back into the item set in order to calculate test-retest reliability of the full summary scales. Intra-class correlation coefficients (ICCs) from these models were used to calculate the standard error of measurement (SEM) and minimal detectable change (MDC) for each CRIS/P subscale. Proxy Satisfaction was compared with the CRIS/P Satisfaction scale using pairwise Pearson correlations and paired t-tests for each item and the summary scores.

Results

Phase II: Pilot testing

Twenty four proxy respondents completed the first survey administration and 23 proxy respondents (100% female), approximately 87% of whom lived with their Veteran completed the second survey (Table 1).

Internal consistency and test-retest reliability

The final CRIS/P measure (Appendix 1) includes 3 subscales: a 48 item Extent of Participation scale, a 51 item Perceived limitations scale, and a 48 item Satisfaction with participation scale. The final Proxy Satisfaction measure includes 48 items. The items that were removed from the Preliminary scales due to poor fit or ICC<0.1 (N=5 items) are shown in Table 2. The items that were removed from internal consistency analysis due to low response rate (3-5 items per scale), but were later added back and thus were included in the analysis of internal consistency and testretest reliability are shown in Table 3. ICCs of remaining items within the subscales (those items completed by more than 6 respondents) ranged from 0.20-0.98, 0.38-0.98, and 0.25-0.87 for Extent, Perceived Limitation and Satisfaction respectively. ICCs for the items of the Proxy Satisfaction scale ranged from 0.58- 0.92. ICCs of the summary scores for each of the CRIS/P subscales were 0.96, 0.95, 0.91 (Table 4) and Cronbach’s alphas were 0.95, 0.95 and 0.96 for Extent, Perceived Limitation and Satisfaction, respectively. MDC 90 was estimated to be 4.7, 6.4 and 6.2 points, for the Extent, Perceived Limitation and Satisfaction scales respectively. The final Proxy Satisfaction scale had a Cronbach alpha of 0.97, an ICC of 0.97 and an MDC of 4.7 points.

Table 2: Items removed from refined CRIS/P scales due to poor inter-item, item to total correlations or ICC<0.1.

Table 3: Items removed only from initial internal consistency analyses due to missingness.

Table 4: ICCs for test-retest reliability, and Minimal Detectable Change of CRIS/P and Proxy Satisfaction Scales.

Comparison of satisfaction scales

Scores of the CRIS/P Satisfaction scale were significantly higher than scores of the Proxy Satisfaction measure for 11 items and the overall summary scores (Table 5). Proxy satisfaction was significantly correlated (r: 0.43-0.86, p<0.05) with veteran satisfaction on 34 of the 47 items (Table 6) and the summary scores were strongly correlated (r=0.78, p<0.0001).

Table 5: Comparison of CRIS/P Satisfaction and Proxy Satisfaction Ratings.

Table 6: Correlations between CRIS/P Satisfaction and Proxy Satisfaction Ratings.

Discussion

We developed and pilot tested a proxy measure of veteran community integration, the CRIS/P that utilized a “proxy-proxy” perspective. Our preliminary analyses showed that the new measure had excellent internal consistency and test-retest reliability. We believe that this measure, when completed by a family or caregiver informant who knows the veteran well, has the potential to provide important, additional information beyond patient self-report to expand upon or clarify the veteran perspective. It is well known that patient reported measures of community reintegration have limitations, particularly for persons who lack insight or have communication or cognitive impairments, a particular concern for persons with head injuries and serious mental illness. It is also recognized that family members and caregivers play an important role in facilitating Veteran reintegration.

Strength of our study was that it utilized caregivers, not clinicians, to develop and pilot test a proxy measure of community integration. Although clinicians and caregivers may have divergent opinions about patient functioning, we believe that the most reliable proxies are persons who spend considerable time with the patient and are familiar with their routine task performance [16]. Compared to such proxies, clinicians have less visibility into the daily life of patients because they do not have the opportunity to observe role functioning and participation in society outside of a limited encounter in the clinical setting. Even when clinicians observe patient performance in the clinic, there may be discordance between their ratings and patient selfreported difficulty in instrumental daily activities such as using the telephone, doing light housework and shopping, preparing meals, handling finances, and managing medications [17]. A study comparing ratings of instrumental activity of daily living (IADL) performance by patients and proxies, found that patients and proxies were more concordant with a criterion measure (inhome performance of IADL) than were clinician ratings based on judgment of patient impairments or observation of performance of tasks in a hospital setting [16]. A systematic review of studies examining proxy-patient agreement on psychosocial functioning demonstrated better concordance between proxies and patients than between clinicians and patients, with a median correlation (studies with N>50) between proxies and patients and clinicians and patients of 0.50 and 0.19 respectively [11].

Although our sample size was modest, we were able to demonstrate that the CRIS/P and Proxy Satisfaction measures had good to excellent test-retest reliability and good internal consistency. However, pilot sample was a convenience sample consisting of all women, and no minorities. The majority of our participants lived with their veteran. Although we had some minority representation amongst participants in the cognitive testing phase, we cannot be certain that our pilot testing results are generalizable to male caregivers or caregivers from diverse racial or ethnic background. Further research is needed to expand the sample to test reliability in a more heterogeneous group of caregivers.

Another limitation of the research is that we did not test the CRIS and the CRIS/P and Proxy Satisfaction measures in caregiver/ veteran dyads and so we do not have an understanding of the relationship between caregiver and actual Veteran reported scores. Although we expect that there will be a moderate relationship between Veteran and caregiver perspectives on community integration, further research is needed to examine the concordance of proxy and Veteran perspectives on community integration as well as the factors associated with discordant perspectives.

We hope that the CRIS-CAT/P and accompanying Proxy Satisfaction measure will be valuable tools for clinical care providers and researchers focusing on post-deployment health, mental health and rehabilitation. These measures are particularly important to the VA given the expectation that veterans and, whenever possible, family members should be involved (with the veteran’s consent) in treatment planning.

Conclusion

This paper reported on the development and pilot testing of a proxy measure of veteran community integration, the CRIS/P and an accompanying measure of Proxy Satisfaction with veteran participation. Analyses support the internal consistency and test-retest reliability of the measures. The CRIS/P consists of 3 subscales: a 48 item Extent of Participation scale, a 51 item Perceived limitations scale, and a 48 item Satisfaction with participation scale. The Proxy Satisfaction measure consists of a 48 item scale. Findings suggest a strong correlation between CRIS/P Satisfaction scale and the Proxy Satisfaction scale, although Proxy ratings of their satisfaction with veteran participation were lower overall and on 23% of items.

Acknowledgements

This research was supported by VA RR&D A9264-S and CIN 13- 419. The information in this manuscript does not necessarily reflect the position or policy of the government; no official endorsement should be inferred. The view(s) expressed herein are those of the author(s) and do not reflect the official policy or position of the U.S. Government.

References

1. Sayer NA, Noorbaloochi S, Frazier P, Carlson K, Gravely A, et al. (2010) Reintegration problems and treatment interests among Iraq and Afghanistan combat veterans receiving VA medical care. Psychiatr Serv 61: 589-597.
2. Defense and Veterans Brain Injury Center (2014) DoD Numbers for Traumatic Brain Injury. Retrieved May 7, 2014.
3. Sayer NA (2012) Traumatic brain injury and its neuropsychiatric sequelae in war veterans. Annu Rev Med 63: 405-419.
4. Galarneau MR, Woodruff SI, Dye JL, Mohrle CR, Wade AL (2008) Traumatic brain injury during Operation Iraqi Freedom: findings from the United States Navy-Marine Corps Combat Trauma Registry. J Neurosurg 108: 950-957.
5. Negrusa B, Negrusa S (2014) Home front: post-deployment mental health and divorces. Demography 51: 895-916.
6. Rodríguez JR, Quiñones-Maldonado R, Alvarado-Pomales A (2009) Military suicide: factors that need to be taken into consideration to understand the phenomena. Bol Asoc Med P R 101: 33-41.
7. World Health Organization (2001) International Classification of Functioning, Disability, and Health Geneva: World Health Organization.
8. Chen KL, Wang HY, Tseng MH, Shieh JY, Lu L (2013) The Cerebral Palsy Quality of Life for Children (CP QOL-Child): evidence of construct validity. Res Dev Disabil 34: 994-1000.
9. Resnik L, Tian F, Ni P, Jette A (2012) Computer-adaptive test to measure community reintegration of Veterans. J Rehabil Res Dev 49: 557-566.
10. Resnik L, Gray M, Borgia M (2011) Measurement of community reintegration in sample of severely wounded servicemembers. J Rehabil Res Dev 48: 89-102.
11. Karches KE, Chung GS, Arora V, Meltzer DO, Curlin FA (2012) Religiosity, spirituality, and end-of-life planning: a single-site survey of medical inpatients. J Pain Symptom Manage 44: 843-851.
12. Pickard AS, Knight SJ (2005) Proxy evaluation of health-related quality of life: a conceptual framework for understanding multiple proxy perspectives. Medical care 43: 493-499.
13. Pickard AS, Lin HW, Knight SJ, Sharifi R, Wu Z, et al. (2009) Proxy assessment of health-related quality of life in african american and white respondents with prostate cancer: perspective matters. Med care 47: 176-183.
14. Ustün TB, Chatterji S, Bickenbach J, Kostanjsek N, Schneider M (2003) The International Classification of Functioning, Disability and Health: a new tool for understanding disability and health. Disabil Rehabil 25: 565-571.
15. White DB1, Cua SM, Walk R, Pollice L, Weissfeld L, et al. (2012) Nurse-led intervention to improve surrogate decision making for patients with advanced critical illness. Am J Crit Care 21: 396-409.
16. Jensen-Dahm C, Vogel A, Waldorff FB, Waldemar G (2012) Discrepancy between self- and proxy-rated pain in Alzheimer's disease: results from the Danish Alzheimer Intervention Study. J Am Geriatr Soc 60: 1274-1278.
17. Gray HM, LaPlante DA, Shaffer HJ (2012) Behavioral characteristics of Internet gamblers who trigger corporate responsible gambling interventions. Psychol Addict Behav 26: 527-535.