So what are fair tests?

Most of us know that it can be a mistake to take a media report of some new medical advance at face value. But the sad truth is that one must also be cautious about reports of treatments even in apparently reputable journals. Misleading and overblown claims about treatments are common, and it is important to be able to assess their reliability.

We run two risks in taking reports of the effects of treatments at face value. We could wrongly conclude that a helpful treatment is actually useless or even dangerous. Or we could wrongly conclude that a useless or even dangerous treatment is actually helpful.

Fair tests of treatments are designed to obtain reliable information about the effects of treatments by

  1. comparing like with like, to reduce distorting influences (biases);
  2. taking account of the play of chance; and
  3. assessing all the relevant, reliable evidence.

This section and the next two sections deal with these three principal features of fair tests.