The Herd Instinct of Mainstream Science

Thomas Gold: New Ideas In Science (Via: Louis Hissink's Crazy World)

J. of Sci. Exploration, Vol. 3, No. 2, pp 103-112, 1989

A motivation which is in a way more serious and more avoidable than the nonlearning one, a motivation that hones out new ideas, is what I brutally call the "herd" instinct. It is an instinct which humans have. It presumably dates back to tribal society. I am sure it has great value in sociological behavior in one way or another, but I think on the whole the "herd instinct" has been a disaster in science. In science what we generally want is diversity -many different avenues need to be pursued. When people pursue the same avenue all together, they tend to shut out the other avenues, and they are not always on the right ones.

If a large proportion of the scientific community in one field is guided by the herd instinct, then they cannot adopt another viewpoint since they cannot imagine that the whole herd will swing around at the same time. It is merely the logistics of the situation. Even if everybody were willing to change course, nobody individually will be sure that he will not be outside the herd when he does so. Perhaps if they could do it as neatly as a flock of starlings, they would. So this inertia-producing effect is a very serious one.

It is not just the herd instinct in the individuals that you have to worry about, but you have to worry about how it is augmented by the way in which science is handled. If support from peers, if moral and financial consequences are at stake, then on the whole staying with the herd is the successful policy for the individual who is depending on these, but it is not the successful policy for the pursuit of science.

Staying with the herd to many people also has an advantage that they would not run the risk of exposing their ignorance. If one departs from the herd, then one will be asked, one will be charged to explain why one has departed from the herd. One has to be able to offer the detailed justifications, and one's understanding of the subject will be criticized. If one stays with the herd, then mostly there is no such charge. "Yes, I believe that because doesn't everybody else believe that? " That is enough justification. It isn't to me, but it is to very many other people. The sheep in the interior of the herd are well protected from the bite in the ankle by the sheep dog.

It is this tendency for herd behavior that is greatly aggravated by the support structure of science in which we believe nowadays. I will read out just one passage here to show that other people than myself have recognized the herd problems: David Michland writes in the REVIEWS OF ASTRONOMY:

I sometimes wonder if the much encouraged and proclaimed interaction among western astronomers leads to a form of mental herd behavior which, if it does not actually put a clamp upon free thinking, insidiously applies the pressure to follow the fashion. This makes the writings of our Soviet colleagues who have partly developed ideas in comparative isolation all the more valuable.

Yes, I have wondered whether one should in fact pursue subjects with a big wall between two groups that are working in the same field, so that they absolutely cannot communicate, and see a few years later whether they come even approximately to the same conclusion. It would then give some perspective of how much the herd behavior may have been hurting. But we don't have that. Even with our Soviet colleagues, unfortunately, we have too much contact to have a display of real independence, to see where it would have led.

This question of how the support of science - and I don't mean only the financial support but also the journals, the judgment of referees, the invitations to conferences, acknowledgments of every kind - how that interacts with the question of herd behavior, is what I will now discuss.

It is important to recognize how strong this interaction really is. Suppose that you have a subject in which there is no clear-cut decision to be made between a variety of opinions and therefore no clear-cut decision to be made in which direction you should put money or which direction you should favor for publications, and so on. No doubt opinions would need a multidimensional space to be presented, but I will at the moment just represent them in a one-dimensional situation.

Suppose you have some curve between the extreme of this opinion and the extreme of that opinion. You have some indefinite, statistically quite insignificant distribution of opinions. Now in that situation, suppose that the refereeing procedure has to decide where to put money in research, which papers to publish, and so on. What would happen? Well, people would say, "We can't really tell, but surely we shouldn't take anybody who is out here. Slightly more people believe in this position than in any other, so we will select our speakers at the next conference from this position on the opinion curve, and we will judge to whom to give research funds," because the referees themselves will of course be included in great numbers in some such curve. "We will select some region there to supply the funds."

And so, a year later what will have happened? You will have combed out some of the people who were out there, and you will have put more people into this region. Each round of decision making has the consequence of essentially taking the initial curve and multiplying it by itself.

Now we understand the mathematical consequence of taking a shallow curve and multiplying it by itself a large number of times. What happens? In the mathematical limit it becomes a delta function at the value of the initial peak. What does that mean? If you go for long enough, you will have created the appearance of unanimity. It will look as if you have solved the problem because all agree, and of course you have got absolutely nothing. If no new fact has come to light and the subject has gone on for long enough, - this is what happens. And it does happen! I am presenting it in its clearest form, and it is by no means a joke. If many years go by in a field in which no significant new facts come to light, the field sharpens up the opinions and gives the appearance that the problem is solved.

I know this very well in one field, which is that of petroleum derivation, where the case has been argued since the 1880's. At the present time most people would say the problem is completely solved, though there is absolutely nothing in the factual situation that would indicate a solution. It is also very clear there that the holding-in that has taken place has been an absolute disaster to research. It is now virtually impossible to do any research outside the widely accepted position. If a young man with no scientific standing were to attempt this, however brilliant he might be, the wouldn't have a hope.

I believe that our present way of conducting science is deeply afflicted by this tendency. The peer review system, which we regard as the only fair way we know of to distribute money (I don't think it is, but it is generally thought to be) is an absolute disaster. It is a completely unstable method. It is completely prone to this tendency; there is no getting out of it. The more reviews you require for a proposal - now the NSF requires seven reviewers for a proposal

-the more you require, the more certain it is that you will follow the statistical tendency dictated by this principle. If you had noise in the situation, it would be much better. There used to be in the United States many different agencies, and there was perhaps an odd-ball over here who gave out some money for one agency, and a funny fellow over there for another. This was a noisy situation, and it was not driving quite as hard towards unanimity. But now we have it all streamlined and know exactly to whom we have to go for a particular subject and, of course, it is an absolute disaster.

Why is it thought that the peer review system would work for science? How about trying to make a peer review system work for other forms of endeavor? Suppose we had a national foundation for the arts, and every painter had to apply to it to get his canvas and his brushes and his paints. How do you suppose that would work? I can imagine some of the consequences, but better than that, we can look them up in historical examples. If you want to read such, in the book The Experts Speak , you can do that. There is a long list of them that you can read - it makes marvelous reading.

Eduard Manet wrote to his colleague Claude Monet, of Renoir: "He has no talent at all, that boy. Tell him to give up painting."

"Rembrandt was regarded as not comparable with an extraordinarily gifted artist, Mr. Ripingill."

William Blake spoke of Titian and the Venetians as "such idiots are not artists."

Degas regarded Toulouse-Lautrec" as merely a painter of a period of no consequence." One wonders how art would have fared in a peer review system.

Or would it be different in music? We can read what was said of Beethoven's compositions by musicians of his time:

"An orgy of vulgar noises" was the verdict of Beethoven's Fifth Symphony by Mr. Spore, a German violinist and composer.

On Tchaikovsky's appreciation of Brahms, "I played over the music of that scoundrel Brahms. What a giftless bastard. It annoys me that this jumping, inflated mediocrity is hailed as a genius." But one could go on almost endlessly with such quotations. Music would not have fared any better.

So we see that the herd instinct is a tendency in the human makeup, which is itself a severe handicap for science. Instead of combating it as best we can, we have arranged a method of nurturing science which actually strengthens it enormously - makes it virtually impossible to depart from the herd and continue to have support, continue to have a chance of publication, continue to have all the advantages that one requires to work in a field.

If in a subject there was initially a diversity of opinions, the review system will assure a very short life for that condition, and soon the field will be closed to all but those who are in the center.

Once a herd is established, by whatever historical evolution this has come about, it obtains such firm control that it is extremely difficult to do anything about it. And even if it were appreciated that that is the situation, one just doesn't know how to interfere. Where then is the right to free speech if every journal has to send each article out to a number of people to review, and the bulk of the people are with the herd? Usually with just one-third of the reviewers very negative, the paper does not get published.

So there is no free speech in the sense that you cannot publish diverse viewpoints. There is also no free speech at conferences because the same is true there. Would all those who have a divergent opinion be able to organize their own conference? Very rarely. We (note: meaning the SSE ) represent perhaps an example here showing that it is possible, but it is pretty rare that one can raise funds to run conferences. Essentially once the herd is established, it will interfere in any one of the activities that one would need to further that science.

Would the Dean of a university be willing to promote somebody to tenure who was outside the pack? He can't, because he has to send out letters to the leading persons in the field - he may inquire from 20 people before he gets permission to appoint somebody to tenure - and how can he get that when the pack is running in another direction than this person? It is absolutely hopeless! So you establish the situation more and more.

Once a herd has been established in a subject, it can only be broken by the most crass confrontation with opposing evidence. There is no gentle way that I have ever seen in the history of science where a herd once established has been broken up.

In many subjects such clear evidence is very hard to come by. In the complex subjects, especially I always think of the earth sciences in this respect, there are always different ways of interpreting any one fact; so many complicated things have taken place that any one fact can have three or four interpretations and the crass confrontation is very rare.

So then when you have a herd, all the money that you spent on it may be wasted, or worse than that, it may actually serve to cement further the bad situation. So it is very likely that money is often spent in science in a way that is absolutely detrimental to that science.

What does the refereeing procedure really look like? How does it really go on? If, for example, an application was made in the early 60's or late 50's suggesting that the person wanted to investigate the possibility that continents are moving around a little, it would have been ruled out absolutely instantly without questions. That was crack- pot stuff, and had long been thought dead. Wegener, of course, was an absolute crack-pot, and everybody knew that and you wouldn't have any chance.

Six years later you could not get a paper published that doubted continental drift. The herd had swung around - but it was still a firm and arrogant herd.

Shortly after the discovery of pulsars I wished to present an interpretation of what pulsars were, at this first pulsar conference - namely that they were rotating neutron stars. The chief organizer of this conference said to me, "Tommy, if I allow for that crazy an interpretation, there is no limit to what I would have to allow." I was not allowed 5 minutes of floor time, although I in fact spoke from the floor. A few months later, this same organizer started a paper with the sentence, "It is now generally considered that pulsars are rotating neutron stars."