Messiness in research

The Guardian has an article Scientists take to Twitter to reveal their less than scientific methods:
"Scientists are a precise bunch. Our experiments are carefully planned down to the last detail, the methods we use are selected with great care and forethought and our sample sizes are perfectly calibrated to ensure statistically valid results. But first our hypotheses are constructed only after carefully reading our peers' work. You can see evidence of this clearly spelled out in any research paper which will invariably present a logical series experiments that lead to a nice clear conclusion all carefully referenced to all the relevant prior-art. So if a reaction was left for 60 minutes there must be a sound scientific reason for this. And of course the equipment we use is carefully built from only the highest quality parts.

At least these stereotypes are what we wanted you to believe in. That is until a couple of days ago. Since then, scientists from all four corners of the twitterverse have not just dismantled that pure-of-thought image but demolished it with repeated 140-character salvos all bearing the hashtag#overlyhonestmethods. Most of these tweets are jokes that rail against the stuffy and sometimes unclear way that scientific papers are written, but there is certainly more than a grain of truth in most of them."

I am reminded of my own messiness in research which pursued me in retirement. In 2003 I had a long paper with Peter Scott after several years of work. We were exhausted and about to give up when we found the final piece and wrote up the paper over an year. Even though the overall picture seemed clear for many years there were too many details and lot of mistakes crept in. Now the lisy of corrections is about thirty pages long, still in progress, and meanwhile simpler approaches appeared. One of the mistakes was the proof of Theorem 7.11 in the paper which was needed for crucial result on the finiteness of some Boolean algebras. I knew the result since 1995 and convinced Peter of my arguments. But a few years ago, Vincent Guirardel pointed out some examples (not counter examples) which suggested that the result may not be correct. After correcting several other mistakes, we finally came to 7.11 and found our proof wrong and could not prove it. Peter started looking for counter examples, I kept saying that I was not convinced that it was wrong. Then the problem started bugging Peter and after several weeks of exhausting work, he found a correct argument a few weeks ago. Though this had a happy ending, the whole process seemed messy and perhaps many of the published results are wrong but may be get sorted out eventually. Anyway, these mistakes made earlier still keep me thinking about some mathematics off and on. It also helps to have brilliant collaborators.