He wrote
"state experimentation, a la welfare reform in the early 90s, could be pretty valuable." "if each of the various state policies were rigorously studied."
I comment.
Yes indeed. Unfortunately, the two clauses together suggests you imagine that state experiments in welfare reform were rigorously studied back in the 90s.
Before going on, let me note my debt to your post "so the costs of welfare reform weren't so low after all" or something, which is one of the main influences on my thoughts most weeks (really).
So what have we learned from a state level welfare reform experiment ? Well we now know that welfare reform kills people
http://angrybearblog.com/2013/06/welfare-reform-kills.html
"research of Peter Muennig, Zohn Rosen and Elizabeth Ty Wilde
From 1994 to 1999, Florida randomly selected a group of welfare recipients into either the Family Transition Program (FTP), ... or the then-standard Aid for Families with Dependent Children welfare program (AFDC“participants in the experimental group had a 16 percent higher mortality rate than members of the control group (hazard ratio: 1.16; 95% confidence interval: 1.14, 1.19; p < 0.01). This amounts to nine months of life expectancy lost between the ages of thirty and seventy for people in FTP."
Odd that this statistically significant result from an actual experiment has had no influence on the debate at all. There is little point having laboratories of democracy if people ignore the experimental results and just go with their prejudices as we do.
Now one problem with using state level experiments is that the devil is in the details and the devil wrote all the details of the national Welfare Reform bill. The original idea from say Clinton (or implemented in Florida under Lawton Chiles) was to offer much more help to people on welfare trying to get off it and also to impose a deadline. The national bill imposed a deadline, made the budget a rigid block grant and left the rest up to the states.
The rigidity was key -- to be able to say that the Federal welfare entitlement was eliminated, the budget was made a function of time. This is why TANF enrollment didn't increase during the recession when, for any sane program design, it would have increased. It also meant that in the late 90s boom welfare budgets were flush. That meant that there happened to be more money for welfare to work assistance. This added to the insane delusion that good outcomes for the poor in the late 90s showed the reform worked (the childless poor did well too, as did the middle class and the rich). Now the reform causes less money for welfare to work assistance, because states can barely keep the poor from starving given the rigid Federal contribution.
In contrast, the Florida bill included massive (and not accidental) increases in help with training, child care, transportation and health insurance. I read and was convinced by, an article about how wonderful it was (in even the liberal New Republic). This makes the demonstrated fact that it killed people much more striking. We know that Floridas welfare reform killed people. We can be quite sure that national welfare reform did too.
But the mere fact that a reform killed Americans is not worthy of any notice (not even here my number one source for info on what went wrong with welfare reform).
I can't quite forgive Bill Clinton for destroying AFDC. It is a crime that it has not been there to help families during the lesser depression.
ReplyDeleteThey had 3200 odd participants, many years later
ReplyDeletenot exactly what you would call a definitive study
(if you actually know anything about this sort of randomized control trial, you know that large positive results are often reported for small samples on the order of a few thousand, results that don't bear up when you do larger studies)
This happens all the time in studies to show the efficacy of a new drug; true, in that case, there is clear motivational bias, but still
Look at how many people they had to enroll in statin trials (S4i wescops) to get a clear all cause mortality benefit...
I don't know about the statistics of doing something more then 10 years after start, but it must have been darn hard to find the right control group after so many years (article is paywalled)
I know a good bit about "randomized control (sic) trials" having published in the peer reviewed medical literature.
ReplyDeleteYou make a claim and provide zero examples. A sample size of one. However, you demonstrate complete ignorance of mathematical statistics. You assert that a failure to reject the null is just like a rejection of the null. You need an example of a significant result which turned out not to be spurious not an insignificant result with a small sample which turned out to correspond to a signficant result.
You provide no zero not one example to support your claim. You assert that it is common for results similar to the one I discuss to turn out to be spurious. I assert that you are bullshitting. I don't expect you to read this reply, but I assert that no such event has occurred in human history.
You need a randomized controlled trial with a sample size on the order of 3,000 with a tiny spurious p value.
The test statistics are based on sample size. I know enough statistics to know that a sample size on the order of 3000 with a binary outcome is large enough that the p-value can't be a result of abuse of asymptotics.
It is possible that the result is due to omitted variables bias. But this is only barely conceivable, because the data come from a randomized controlled trial.
I think your comment demonstrates that you are incapable of accepting facts which contradict your ideology. Also that you don't know jack about mathematical statistics and are innumerate.