When science fails

Progress in science is a matter of developing theories that are better than the existing ones. A new theory is better if it explains phenomena that the old theory fails to explain, although they are within its range and should have been explained by it. Or the new theory gives a more plausible and simpler explanation of the facts. Of course, with the help of some additional suppositions it may be possible to give the old theory a new or broader basis, but with each additional supposition the old theory becomes more complicated and by that more unlikely. The rule of thumb in science is the simpler the better and generally it works that way. So, many old theories have quitted the scene in exchange for new ones that played their parts better. Now we know that the sun doesn’t orbit around the sun but that the opposite is the case. We know that there is no ether that fills empty spaces but that a vacuum is really possible. And we know now that life cannot originate from dead matter. How progress works in science has been summarized by Karl Popper in a well-known scheme:

P1 > TT > EE > P2

In words: If we have a problem that we can’t explain with the old theory (P1), we revise it or develop a completely new theory (TT). Then we perform experiments (EE) in order to test our new ideas, and if the new ideas are confirmed, the old theory has been improved or replaced in favour of the new one. However, usually we see then new problems (P2) and the cycle starts again. Since here in my presentation we start with the idea of a theory that appears to have mistakes, we might describe the process of theory evolution also this way:

TT1 > P > TT2 (tentative) > EE > TT2

Since TT2 is better than TT1 Popper talked here of error elimination. This implies that actually the old theory has failed, because it contained mistakes. But this is how scientific progress works and there is no other approach. And often it is so that the old theory has been used for a longer or shorter time to everyone’s satisfaction, despite its mistakes.

TT1 can also fail for another other reason: Not because it has been replaced by a better theory but because it is bad science. That’s what we have seen in my last blog. If we look at the first scheme, then the trouble is not that P that can’t be explained by TT but we think that TT has explained P but the explanation is false. It is because the tests that seem to substantiate TT have not been well done. So there is a failure in the performance of EE. This can happen by accidental occurrences, but usually it is so that the methods of investigation have not been well applied by ignorance, negligence, lack of money or something like that, or in order to please the one who pays the investigation or even by fraud. In other words, the tests failed because of bad science. If it happens now and then it’s a blot on science. However, if it happens too often and on a large scale, sooner or later it will lead to a crisis. Then the schemes are no longer as shown above but they have become:

P1 > TT > EE > P2

and

TT1 > P > TT2 (tentative) > EE > TT2.

When the failures come to light we have a crisis in science and the schemes as just represented stand for regress in science. And that’s what we see now in social psychology. From a methodological point of view the rules are simple: Besides a strict and correct application of the methodical prescriptions any investigation has to be replicated, if possible by other investigators in another setting and with another set-up. But often there are many reasons not to replicate an “old” experiment. Practical reasons, financial reasons and human reasons, for little credit can be gained by affirming what has already been said by others (and if the original investigation has been well performed, this will be the result). But a crisis in social psychology does not hurt only social psychology itself (or any other science that is hit by such a crisis, as the case may be), but it has wider consequences. Results are applied. Convinced that the psychological investigations have been well done, therapists have put their outcomes into practice. In philosophy they have affected the view on man. Etc.

The key question is how it could come that far. If we don’t give it an answer it can happen again, in psychology and elsewhere. There is a name for the present problem: Replication crisis. It’s a name that points already to its solution. But who will be prepared to tackle the problem, as usually it is so that it doesn’t bring much to you if you do old wine in new bottles?