Measures that matter

Measures that Matter

The Center is working to identify specific metrics that can be used to increase the specificity of measurement in specific situations, and to develop “measures that matter”, especially those that capture the real value of primary care.

Measures that matter

Measures that Matter

The Center is working to identify specific metrics that can be used to increase the specificity of measurement in specific situations, and to develop “measures that matter”, especially those that capture the real value of primary care.

Situation

Value-based payments to primary care frequently employ measures that are misaligned with the high-value functions of primary care. Current measure sets presume that quality of care is the sum of quality measures for individual diseases. Financial incentives drives behavior to maximize these low-value measures at the cost of high value functions, resulting in tremendous opportunity costs, increased clinician burnout, and diminished value of primary care for people and populations.

Background

Well-designed and supported primary care is an important source of improved outcomes in high performing health systems even though it may produce lower disease-based quality measures. This has been called the Paradox of Primary Care. The United Kingdom went down a similar disease measurement path with the Quality Outcome Framework 15 years ago and have since largely abandoned it due to burnout and lack of population health improvement. This does not mean that measures or payment schemes that use them are bad, but it does suggest a need for better alignment between measures that matter and providing sufficient resources to address them.

Resources

Publication in the Annals of Family MedicineHave more questions? Email [email protected]

Assessment

The Center for Professionalism & Value in Health Care aims to assess and promote the most meaningful measures in several health care sectors starting with primary care. The Center aims to produce comprehensiveness measures (highest quartile has 15% lower total costs and 25% lower hospitalization rates),continuity measures (highest quartile have 25% lower total costs and 25% lower hospitalization rates), as well as total cost of care and low-value care measures. Based on this research, CMS has approved the latter for use in our QCDR as a MIPS measure. The American Board of Family Medicine Foundation funded research with the Larry A. Green Center that produced thePatient Centered Primary Care Measure, a PRO which won the National Quality Forum’s (NQF)2019 Patient-Reported Outcomes Abstract Award and has been fast-tracked for NQF and CMS endorsement. The research underpinning this PRO demonstrated close association with continuity and comprehensiveness, and strong endorsement by both patients and providers. The Center will continue to develop and study these measures using the PRIME Registry as a testbed, and using claims data to assess them across most family physicians, training programs, and health systems. JAMA and Annals of Family Medicine published our studies done in collaboration with the Robert Graham Center showing that physicians’ cost-related behaviors are highly correlated with where they trained and last for 15+ years after training. These measures not only have utility for clinicians, but for identifying training programs and health systems that are not supporting high-functioning primary care.

The Center’s aims in developing Measures That Matter for primary care practices is to better align assessment and payment policies with what patients and clinicians know to be valuable, to reduce burden, and to reduce burnout.

Learn more about the Center for Professionalism and Value in Health Care

The Person-Centered Primary Care Measure, Now Available for Use

PRIME 133, a Person-centered Primary Care Measure is now featured in the PRIME Registry Measure Set, and available for use as a MIPS (Merit-based Incentive Payment System) measure.

What is PRIME 133?

PRIME 133 is a patient-reported measure of exemplary primary care that has been developed by the Larry A. Green Center, based on extensive developmentwork with patients, clinicians and health care payers. The measure is the winner in the Patient-Reported Outcomes category of the National Quality Forum (NQF) Next-Generation Innovator Abstract Award.

The Person-Centered Primary Care Measure (PCPCM) focuses attention and support on the integrating, personalizing, and prioritizing functions that patients and clinicians say are important. A measure based on these principles may reduce both the de-personalization experienced by patients, and the measurement burden, burnout and crisis of meaning experienced by clinicians.

Implementation

The PCPCM uses a survey to ask patients to assess 11 distinct yet highly interrelated items regarding their assessment of the care they receive. The 11 items were developed with input from hundreds of patients and physicians, and are associated with better personal and population health, equity, quality and costs.

Potential Impact

The work that participating practices do will help support efforts to gain NQF and CMS endorsement for use of the PCPCM in payment, certification and regulatory programs.

Step 1: Information Gathering

Measure development begins with information gathering—environmental scanning, developing a business case and engaging stakeholders.

Step 2: Draft Specifications and Begin Testing

After the information gathering phase, we begin to draft the measure specifications and begin initial testing and feasibility studies.

Measure specifications provide the comprehensive details that allow the measure to be collected and implemented consistently, reliably, and effectively.Measure technical specific actions will address the following questions: How will the measure be named? Does the name mean anything to people when they read it? Do they understand what that measure is about? What would the setting of the measure be (e.g., ambulatory office)? How will the data be collected? All of these questions have to be answered before testing begins.

Harmonization is all about reducing burden. Look at measures currently in practice and determine if there are places where our measure could be harmonized with the existing measure(s). May include age ranges, performance time period and allowable values for medical conditions or procedures, code systems, descriptions.

Measure Testing means testing quality measures, including the components of the quality measure such as the data elements, the scales (and items in the scales if applicable), and the performance score.

There are two parts to measure testing: alpha and the beta testing.

Alpha testing helps identify early issues and often begins as early as the conceptualization step and is repeated during the development of the measure specifications.

Beta testing, which is also referred to as field testing, generally occurs after the initial specifications have been developed, and strives for representative sample sizes –multiple sites/settings. The primary purpose for beta testing is to understand the usability of the measure and to test the scientific acceptability of the measure.

After the testing ends, the results are analyzed with a return to the specification phase, or even the conceptualization phase, to rework the measure before testing again.

Step 3: Measure Implementation

The measure is then submitted for NQF endorsement and for use in the 18 CMS quality reporting and payment programs. This process can include comments and feedback submitted through the Pre-Rulemaking or Rulemaking processes or through ad hoc comment processes. Eventually, proposed measures are published in a proposed rule as part of the rule making process. Generally, each rulemaking program releases a proposed rule through the Federal Register once a year around the same time period. After reviewing the comments, CMS will decide whether the proposed measure should be implemented in the program. If so, a final rule will be published specifying the measure, the program the measure will be implemented in and the implementation date.

Step 4: Re-evaluation

The regular reevaluation of measures is vital as the science and other factors are always changing (e.g., development of new clinical guidelines, new technologies for data collection, discovering a better way to calculate measure results). Continually reviewing the measure will ensure it remains relevant and meaningful. Measures that stop being useful are retired.

What’s the difference between submitting to CMS versus submitting to NQF?

The National Quality Forum (NQF) submission is about the endorsement process where a consensus-based entity reviews the measure using five evaluation criteria to assess the measure on its own merit and independent of a CMS program. It essentially gives it that stamp of approval, and so endorsement/NQF submission is separate from CMS implementation. NQF endorsement is valued for measures in CMS programs, but it is not a requirement.

The CMS implementation process takes the measure from being in development to being actively used in a quality reporting or payment program (QRP/QPP).

The Five Phases of Measure Development

Conceptualization: Develop measure concepts and then narrow down to specific measures. The developer conducts an environmental scan and requests input from a broad group of stakeholders, including patients.

Specification: Identify the population, the recommended practice, the expected outcome and determine how it will be measured.

Testing: Assess the suitability of the quality measure’s technical specifications and acquire empirical evidence to help assess the strengths and weaknesses of a measure.

Implementation: Identify measures to submit for the CMS selection and rollout processes, adopt measures into CMS programs, and seek endorsement.

Use, Continuing Evaluation, and Maintenance: Ensure that the measure continues to add value to quality reporting measurement programs and that its construction continues to be sound.

Decision Criteria

The following decision criteria throughout the measure development cycle ensures a measure meets the applicable standards before moving to the next phase:

Importance to measure and report—including analysis of opportunities for improvement such as reducing variability in comparison groups or disparities in healthcare related to race, ethnicity, age, or other classifications.

Scientific acceptability—including analysis of reliability, validity, and exclusion appropriateness.

Feasibility—including evaluation of reported costs or perceived burden, frequency of missing data, and description of data availability.

Usability—including planned analyses to demonstrate that the measure is meaningful and useful to the target audience. This may be accomplished by the TEP reviewing the measure results such as means and detectable differences, dispersion of comparison groups, etc. More formal testing, if requested by CMS, may require assessment via structured surveys or focus groups to evaluate the usability of the measure (e.g., clinical impact of detectable differences, evaluation of the variability among groups).

Our Partners

We are grateful for the collaboration and support of our current partners in this initiative.

  • The Larry A Green Center
  • The ABFM Foundation
  • The PRIME Registry
  • Virginia Commonwealth University

Interested in opportunities to collaborate or support this initiative? Contact Jill Shuemaker

Related Publications

A New Comprehensive Measure of Primary Care