Non RANDom panel attrition update

Here I enthuse about a criticism by John Nyman of a RAND study on the effect of copayments on demand for health care and health outcomes.

Still following in the footsteps of a kid half my age, I note that Ezra Klein notes that RAND has responded to the criticism with this pdf.

The RAND study was a genuine experiment in which participants were randomly assigned to complete coverage or one of 6 different plans with copayments. RAND concluded that demand for health care was reduced by copayments but found significant worsening of health outcomes only for some poor participants.

The original Nyman critic noted that participants with less generous programs were much more likely to switch back to their original insurance plans (as they were allowed to do). If sicker people did so, this could bias the result.

RAND attempted to follow the health outcomes of people who switched out of the experimental plans and has data on health outcomes of 77% of them. Thus the attrition problem is much smaller than I thought after reading Nyman's abstract (I would have had to pay to read his actual article).

I don't understand the response on one point

"at the end of the
Experiment we were able to collect health status data on 77 percent of those
who had left the Experiment prematurely (85 percent of those who survived). "

I assume "survived" is meant in the normal sense as in "did not die". All participants who died while participating in the experiment were considered to have left the experiment prematurely. Death rates did not differ significantly across plans (the deaths were so rare that one wouldn't expect them too). From the response I can't figure out the death rates of people (if any) who left the financial part of the study, then died. This is an important number.


Just thought I should say that.

I am very struck by the following paragraph in the reply

4. Moreover, Nyman’s speculation about a high degree of non-random attrition
is contradicted in work that is unpublished but also posted on Newhouse’s
home page. In this work Manning, Duan, and Keeler carried out additional
analyses of those who did not complete the Experiment. They concluded that
there was in fact a modest amount of non-random attrition, but that its effects,
if accounted for, would have left our conclusion that cost sharing reduced use
unchanged.

This is striking because there is no claim that accounting for non-random attrition would leave the conclusion that the health effects of reduced use of care due to copayments were negligible.

The web page is here

the *.pdf by Manning Duan and Keeler is the third article from the top on the web page.

I quote from Manning Duan and Keeler

"we found that the dropouts
had significantly different rates of use, but that the magnitude of the difference was small."

"Dropouts were sicker on average at enrollment than those who stayed,"

The last statement is very very surprising in an article cited in a reply to the assertion that the vastly greater number of dropouts from the plans with copays biased up the average health status of people assigned to those plans.

In fact, the reply seems to be based on the argument that 77% = 100% which is not exactly true is it ?

Furthermore, it is not certain that the loss of information on the remaining 23% of dropouts was itself random. People who are hard to track may be less healthy than people who are easy to track.

The sign of the difference in care received by dropouts and continuing participants in the period before dropouts dropped out of the plans is surprising. In particular there was a significantly lower rate of inpatient care for participants in the "family plan" before they dropped out. Participants were paid a fee while they participated. I have a hypothesis. What if people with copays delayed going to the hospital for a while, even though they knew hospitalization was necessary, in order to collect the fee, then dropped out. People planning to stay in the program would have little incentive to do this as they would have to pay the copay eventually anyway. People planning to drop out would have a strong reason to delay care until they dropped out. Dealying hospitalizaion does not seem to me to be a good approach to saving on health care.