Innumeracy in the New York Times

Great wonderful important cancer treatment news slightly blurred by an odd approach to statistics of very small samples.

Compare and contrast the words and the numbers
Words

"Stories about such rescues from death's door have given hope to tens of thousands of cancer patients who have tried Iressa, ...

At the same time, patients without the mutation might forgo Iressa, saving them or the health care system $1,900 a month for a drug that is not likely to help them. Still, some of the researchers said that the findings were based on small samples ...

"The question then is what if you don't have this mutation? I don't think there's enough information to say you shouldn't get the drug.""

Numbers

"One research group found a mutation in all five of the tumors it checked from patients who had responded to the drug and in none from the four patients who had not responded. ...

The other study found the mutations in the tumors of eight out of nine responders, ... and in no tumors of seven nonresponders. "

So of 12 people with cancer and without the mutation one responded to Iressa. That's one out of 12. The article doesn't say what kind of cancer that patient had (from context I guessed lung cancer). A one out of 12 performance in treatment of patients where conventional treatment failed is outstandingly excellent. It is not as good as the 13 out of 13 performance of people whose tumors had the mutation, but it is crazy to even discuss the possibility of not giving the drug to people who lack the mutation based on a sampel with 1 out of 12 success.

The 1 out of 12 isn't the only data useful for estimating what fraction of cancers without the mutation respond to Iressa. The calculation of the overall response rate from a large sample (about 10%) and the fraction of tumors with the mutation 16 out of 103 are also useful information. After some complicated (and probably incorrect) calculations, which I will not bore myself typing up, I get a maximimum probability estimate that a cancer without the mutation will respond to Iressa of 1 in 16 which is still excellent.

Now if 0 cancers without the mutation had responded, the point about how the sample is so small that this does not reject the hypothesis of a clinically significant probability of response would have been valid. Clearly it was perfectly possible that none of the 12 treated patients with tumors without the mutation would have gotten lucky. In a sample of 12, one success makes a huge difference in the point estimate of the probability of success. Ignoring this case in the discussion is crazy.

Anyway ANDREW POLLACK and the doctor he quoted very briefly (perhaps deleting reasoning like the above) reach a reasonable conclusion. The article should be very prominent because it is important to spead the great news as fast as possible.

[update]
Ransom M Wardell writes

"Response rates were 18.4 and 11.8%, and disease control rates were 54.4 and 42.2%, " This means that the roughly 10% was very very roughly (and shows further disinterest in numbers even if they are very important numbers). It also reminds me that response and cure are different. I don't know how long people have to do OK to declare their disease controlled. 1/12 or 1/16 times 42.2% = still great for non small cell lung cardinoma.