An expert in quality measurement offers guidance for physicians seeking to establish a process for assessing how they provide behavioral care.


Harold Alan Pincus, MD, is professor and vice chair of the Department of Psychiatry and co-director of the Irving Institute for Clinical and Translational Research at Columbia University and director of quality and outcomes research at NewYork-Presbyterian Hospital.

One of his specialties is research into incorporating quality measures in behavioral care, with articles published in JAMA, Health Affairs, and the American Journal of Psychiatry, among other journals.

In this article, Pincus discusses what physician leaders should know when seeking to use metrics to guide the integration of behavioral health care and primary care.

Q: Why are quality measures important in behavioral care integration?

One key reason is the costs involved in caring for people with comorbid behavioral health and general medical conditions. Many studies have made it clear that if you look at the highest-cost patients in any health system, there is a high prevalence of mental and addictive disorders. If you look at the highest-cost segment of Medicaid, as shown by a recent report from the Center for Health Care Strategies, mental health issues are almost universally involved. If you look at the top 10 sources of 30-day hospital readmissions for Medicaid, four of 10 are mental health conditions. Depression is the second-leading source of disease burden in the world, according to the World Health Organization. Among sources of disability in people ages 15 to 45, the most productive age group, mental disorders are four of the top five.

Of course, there are other issues besides costs. Quality measures help patients know whether they are getting quality care and let providers know whether they are providing quality care, and how they stand compared to other providers. And at the policy level, as we move into value-based purchasing and other initiatives, and away from paying on the basis of price and volume, we need to add quality into the mix. So how do we quantify quality?

When you think about it, there are different kinds of measures for different purposes. Some are primarily for improvement—they help healthcare providers and organizations and health plans figure out how to do a better job. Because there’s a saying that it’s hard to improve unless you can measure things. “If you don’t know where you’re going, you might wind up someplace else”—the whole Yogi Berra thing.

Then there is the whole push in the policy area to really do quality measurement for purposes of accountability. There are various incentives tied to that, whether it’s public reporting or money.

Q: What form should these measurements take?

Basically there are different categories, or buckets, that people talk about when they talk about measures. The traditional categories are from the Donabedian Model (from physician and health services researcher Avedis Donabedian)—structure, process, and outcomes.

Structural measures assess the capacity to provide high-quality care, for example trained personnel, information technology, and facilities; process measures look at the utilization of evidence-based practices; and outcomes measures look at whether people are actually getting better. Within each of those buckets there are different domains that one would look at, depending upon the particular goals, settings, and clinical context.

Although some organizations place outcomes at the highest level, given some of the issues involved in measuring outcomes (such as risk adjustment), you ideally want a balanced portfolio of measures across multiple domains.

There are other ways to categorize measures as well. A 2001 report from the Institute of Medicine, Crossing the Quality Chasm, categorized measures in terms of the goals of safety, effectiveness, patient centeredness, timeliness, efficiency, and equity.

So how does one apply these measurement concepts to mental health and substance abuse? One aim is making sure there is some kind of balanced portfolio of these things. And there has been an evolution here—a recent report identified 510 measures addressing behavioral health that have been developed by various groups.

Q: So what should the measurement process look like?

Ideally the measurement process starts with an evidence base—evidence from trials and studies that shows what works in particular conditions and settings. That evidence then gets compiled through various mechanisms, generally by professional associations, and is translated into guidelines. And depending on the amount and quality of the evidence, the guidelines are more or less specific.

One issue is that there are gaps in the evidence base. Even when there is reasonable evidence, it’s often hard to get specific because the next step is operationalizing those guidelines into measures that can be easily and reliably measured.

For example, let’s say you have a recommendation that individuals who have major depression should receive either antidepressants or psychotherapy as treatment; how do you operationalize that statement into a measure? You have to create a reliable numerator and denominator. The denominator is people who have depression, but how do you measure that? Is it individuals who have claims data for at least one encounter with depression, or do you need to be more specific? And is a single claim for depression sufficient? How do you know they actually met the criteria for depression?

And how do you develop the numerator? What does it mean when you have data that they received an antidepressant or underwent psychotherapy? You can measure the receipt of medication, but you don’t know whether they actually ingested the medication. And for psychotherapy it’s even more complicated—you might know they had a CPT code of 45 minutes of psychotherapy, but you have no idea about the content of that psychotherapy. Does it match what was tested in the studies in the literature on which the guidelines were based?

Q: Please walk me through an example of a measure for the integration of behavioral health care with primary care.

The clinical strategy for managing behavioral health problems should be the same as is used for diabetes and hypertension, so the measurement structure should be similar as well. Basically you have to systematically, longitudinally measure whether the patient is getting better or worse over time. This measurement-based care approach involves more action-oriented measures, so if a patient isn’t getting better, you would augment the treatment from a menu of reasonable care options. And you relentlessly follow up to make sure the patient doesn’t drop out, which is another problem that occurs when you’re dealing with patients who have behavioral health problems.

What’s increasingly happening is this kind of approach is being built into the expectations for all chronic illness care through the development of registries that track patients. Some of these registries are built around particular conditions, and others track multiple conditions.

So a structural measure that can be applied to this situation might ask the following questions: Do you have a system that enables you to apply a measurement-based care approach? Do you have a patient registry? Do you employ care managers?

And then the process measure in this case might be: Are you seeing your patients frequently enough, systematically assessing them with a standardized measure—for example, the nine-item Patient Health Questionnaire (PHQ-9)—and getting that data into the registry?

And finally, the outcome measure would be: Are your patients actually getting better? Of course, one of the issues with a true outcomes measure is that you need to stratify the risk, otherwise you penalize a healthcare system that sees a more sick population or reward a system that cherry-picks certain patients.

Q: Has progress been made recently regarding measures?

Yes. CMS [the Centers for Medicare & Medicaid Services] recently added two measures for depression that accountable care organizations [ACOs] must collect. Previously there were 33 measures that ACOs had to collect, and one was depression screening and follow-up. They added two outcomes measures for depression: remission and/or clinically significant improvement. Those measures require the collection of standardized data using the PHQ-9.

Q: What are some of the barriers to improving the measurement process of the integration of behavioral care with primary care?

As I mentioned, one issue is that we have a lot of gaps in the evidence base—we don’t always know a lot about what works and what doesn’t. Because good measures start with a solid evidence base, that can be a problem.

Another issue is that there are no lab tests for behavioral health. There are fairly good measurement tools, but they are not necessarily captured in claims data or other routine clinical data, and they are not used by providers systematically.

There are attempts to remedy this. For example, in Minnesota, there is a program called the DIAMOND [Depression Improvement Across Minnesota, Offering a New Direction] Program, which is applying a measurement-based care methodology for incentivizing medical groups to treat depression.

Another approach to assessing integrated care is to apply measures for general medical conditions such as diabetes or hypertension or for preventive health such as immunizations and screenings and stratify the population to see if there are disparities between individuals with comorbid mental disorders and those without comorbidity. This type of comorbidity is prevalent and highly clinically significant. People with severe mental illnesses die about two decades earlier than the general population.

Q: What are the next steps in improving the measurement of behavioral health care?

It’s important to know that the majority of people with mental health issues are treated in the general health sector, not the mental health sector. So we need to think about using the measures in a way that holds generalists and mental health specialists equally accountable for outcomes. It’s the concept of shared accountability. If someone has schizophrenia and diabetes, the primary care physician who takes cares of their diabetes should also be accountable for the schizophrenia. And I, as the psychiatrist, also should be accountable for the outcomes of both schizophrenia and diabetes. This means we have to talk to each other.


Ed Avis is a freelance writer in Chicago who specializes in healthcare-related topics.

Interviewed for this article: Harold Alan Pincus, MD, professor and vice chair of the Department of Psychiatry and co-director of the Irving Institute for Clinical and Translational Research at Columbia University; director of quality and outcomes research at NewYork Presbyterian Hospital; and a senior scientist at the RAND Corporation.

Publication Date: Tuesday, May 03, 2016