Sunday, November 21, 2010

A call to publish negative results?

The Journal of Cerebral Blood Flow and Metabolism has taken the brave and, I would argue, constructive step of actively soliciting manuscripts that present negative results. In their words:
"In addition to original research articles, authors are welcomed to submit Negative Results. The Negative Results article intends to provide a forum for data that did not substantiate the alternative hypothesis and/or did not reproduce published findings."
Good for them! With one of the biggest physiology journals getting its act together, what field might be next? How about neuroimaging?

If any field could use a forum for negative results it is neuroimaging, and fMRI in particular. We seem to have a bias towards positive results that is second only to pharmaceuticals research. Of course, it is perfectly natural for scientists to want to find something rather than not find it. Not finding an earth-like planet orbiting another solar system isn't nearly as exciting as finding one. And one wonders whether Ponce de Leon would have got tenure at a modern university if his most cited work was entitled "On not finding the Fountain of Youth."

One of the challenges facing fMRI is that it demands extensive statistics or modeling to coax meaning out of tiny signals in an ocean of noise. The convoluted analyses provide skeptics with plenty of ammunition that we are basically making it all up by establishing the test that yields the answer we were looking for. (I have a former colleague - a 'real' MR scientist - who claims dismissively that fMRI stands for fictional MRI.) Without rigorous stats/models, though, it is easy to fall into the trap of false positive errors, a problem that has led to accusations of all sorts of voodoo of its own. How, then, should we treat negative fMRI results? Are there any caveats to encouraging their publication?

A negative result or a bad experiment: what's the difference?