John D Peipert, Julie A Brown, Mike Cui and Ron D Hays
To date, no published study has compared modes of survey administration for the Consumer Assessment of Healthcare Providers and Systems In-Center Hemodialysis (ICH-CAHPS®) survey. We conducted a randomized experiment comparing response rates, item missingness, reliability, and mean scores between phone-only administered and mixed phone and mail (mixed mode) administered ICH-CAHPS surveys. Patients were randomized to be administered the ICHCAHPS by telephone only mode (n=1,781) or mixed mail and telephone mode (n=1,362). Since some patients did not complete the survey in the mode they were randomized to, analysis was conducted as intent-to-treat [(ITT); as randomized] and as actually completed. The response rate was significantly higher (p<0.001) for mixed mode (56%) than for telephone only mode (39%). Small differences in internal consistency reliability were observed; e.g., the Nephrologist Communication/Caring composite differed slightly between mixed (α=0.90) and telephone-only (α=0.88) modes (p<0.01) (ITT analysis). ICH-CAHPS means were higher (more positive) for mixed mode compared to telephone-only mode for 2 of the measures: Doctor Communication and Caring (77.8 vs. 74.7; p=0.02) and Global Doctor Rating (84.7 vs. 80.7; p<0.001). However, there were also significant differences in the opposite direction (telephone-only more positive than mixed mode) for the Global Staff Rating (85.6 vs. 83.5; p=0.03) and Global Center Rating (88.4 vs. 83.4; p<0.001) measures (ITT analysis). Dialysis administrators can use these results to maximize response rates to ICH-CAHPS surveys, and public reports of ICH-CAHPS should adjust for mode of administration.
