Reporting in a World of Multiple CAHPS Measures Mark Schlesinger, Yale University David Kanouse, RAND September 9, 2012

Challenge 1: Availability of measures with overlapping content creates potential for confusion in reporting Challenge 2: Proliferation of domains to be reported creates a demand for “roll-up” measures that summarize across domains Overview 1

Example: The Clinician & Group Surveys produces the following four measures: –Getting timely appointments, care, and information (5 items) –How well providers (or doctors) communicate with patients (6 items) –Helpful, courteous, and respectful office staff (2 items) –Patients’ rating of the provider (or doctor) (1 item) Challenge 1: Overlapping Content 2

Supplemental item sets designed for the Clinician & Group Surveys have additional composite measures and individual items that relate to the core domains; e.g., –Getting timely appointments through or a website (HIT composite, 1 item) –How well providers communicate about medicines (Health Literacy composite, 4 items) Challenge 1: Overlapping Content 3

New composites and items that address specific topics but also fall within the domains covered by core composites create additional challenges for labeling of composites and reporting –Labels need both to describe content of composite and distinguish it from other composites –Potential for consumer confusion and overload Overlapping measures also create a temptation for sponsors to create their own (non-standard) meta-composites Challenge 1: Overlapping content 4

Do not publicly report items not recommended for reporting; these items are for QI purposes When testing reports with consumers, examine their understanding and interpretation of composites in relation to other potentially related composites, not merely in isolation Don’t even think about building your own in- house meta-composite Challenge 1: Recommendations 5

As the number of performance measures grows, integrating that information becomes more difficult for consumers Example: The Hospital CAHPS Survey contains six composite measures and three individual questions covering highly diverse domains (e.g., communication with nurses and doctors, pain management, discharge information) Challenge 2: Roll-up measures 6

Integrating information across multiple domains to arrive at an overall evaluation is difficult, and, use of roll-up measures offers a way to make that task easier Use of roll-up measures also tends to increase the number of domains that are explicitly weighed in the decision However, roll-up measures have disadvantages as well as advantages Challenge 2: Roll-up measures 7

Roll-up measures may provide consumers with an easy way to make a decision without understanding the dimensions on which options vary Without some kind of tailoring based on consumer preferences or characteristics, roll-up measures tend to be “one size fits all” Challenge 2: Roll-up measures 8

Some of the drawbacks of roll-up measures may be offset by offering them as a tool along with more detailed measures Providing roll-up measures on a website makes it possible to tailor the roll-up to an individual’s preferences, e,g., by asking which domains should be underweighted or dropped from consideration Research on the best ways to use roll-up measures in public reports on quality is needed Challenge 2: Roll-up measures 9