The Price of Objectivity & Acceptance: The Art of Cherry Picking in Science.

Cherry Picking Disguised as Scientific Method

There has always been a tendency not to include results that did not fit expectations. Some researchers would try to be “fair”, by removing a ‘positive’ data point also when they removed the ‘negative’ one. However the practice is not statistically valid, as it skews towards expectations and away from empirical data.

Many of the methods used may have been fairly innocent by intention, but contributed to a distortion of the results, or more significantly masked or hid results that were not artifacts at all but data points that provided information that questioned the assumptions or supported a more complex understanding of the phenomena.

Occam's Razor undoubtedly also became a factor that influenced the removal of unexpected data points. It emphasized the simpler explanation to avoid superfluous complexity. Unfortunately, the simpler explanation was not always the correct choice.

When data is cherry picked, under the pressure to get results, it delays science and creates an obstacle for science to be objective. Research that follows, which ironically could  corroborate the un-manipulated results, now faces greater scrutiny, because it contradicts already accepted and published results.

Cherry Picking and the Money Trail

Unfortunately, in modern times, much of medical research is cherry picked. Some estimate that the amount of misrepresentation, in one form or another, is as great as 50%. It is one reason why drugs go on the market that are less effective than claimed or have serious side effects that were not initially revealed, or are revealed too late, or become public knowledge because of the litigation that ensues.

With respect to how researchers may mold or cherry pick data, much is revealed when you follow the money trail. If a researcher is sponsored by a pharmaceutical company that is eager to have their product approved, then there is considerable pressure to get positive results, and as a consequence under report negative results. There are a number of ways that the data or research can be deliberately skewed or manipulated to get the results being sought.

Once simple method is for the researcher to deliberately construct several parallel studies, and only report the positive ones. It is a fairly easy tactic to employ. For example, a researcher may decide to freeze or discontinue the studies that hold negative or not sought after results, and either ignore them, declare them as faulty, contaminated or simply disclose that these studies are still ongoing and it is too early to report results, and only the ‘completed’ studies are being reported.

Cherry Picking in Publication

Aside from the cherry picking that goes on in research, there is also the cherry picking that occurs in publication. Publishers of peer reviewed journals find themselves in the situation that their own authority or reputation as a respected publisher is an important factor in determining both the quality of submissions and what they can charge for publication copies. When journals receive research papers that are controversial or present evidence that questions previously held beliefs or theories, they are often treated very differently, as the example with the publication Nature shows. Instead of the normal peer review process, these are either ignored or when they can not be ignored due to the significance of the research or prestige of the researcher, the path is often chosen to discredit the researcher, or find a catch or loophole in order not to risk the reputation of the journal itself, if it were to publish, or risk that the submission is accepted through peer review.

The Consequences of Cherry Picking

Of course, intentional cherry picking can skew the results and promote an incorrect conclusion, or an incorrect confidence level for the data being presented, and this behavior would be rightly considered fraudulent, when it is uncovered is treated as such.

More common has been the practice of removing data points that were not consistent with expected observations. It is not uncommon to re-examine older research and find that in ‘cleaning up’ or removing inconsistent ‘artifacts’, evidence that would have pointed to new paradigms was buried and this delayed progress.

The consequence of cherry picking is that it tends to promulgate itself. If previous publications have cherry picked, then it is more difficult to convince peer reviews that your own observations are accurate or valid without doing the same.

Beyond Cherry Picking

There are other kinds of cherry picking that effect larger scopes of research. Examples abound in archeology. For decades, artifacts have been discovered by reputable investigators, or authenticated with signed statements verifying discoveries, that were ignored because they did not fit the beliefs of the day. Even whole structures were unearthed and although invitations were consistently offered for researchers to investigate the evidence, no one came.

A case in point were the discoveries of pottery and other artifacts in the Andes that were dated to be tens of thousands of years old. They were ignored by mainstream archeology, because of the belief that all of the indigenous populations of North and South America came from the Alaskan-Bering Land Bridge during the last part of the Ice Age, about 12,000 years ago. This theory was supported by the absence of sites older than that date in North America.

More recent excavations in the southern parts of North America and in South America have shown that there were in fact civilizations dating back at least 34,000 years, and that migrations extended from Asia over several ice ages.

Winston Churchill is quoted as having something to say about this. He is quoted as saying: “Men occasionally stumble over the truth, but most of them pick themselves up and hurry off as if nothing ever happened.”

When researchers encounter something that fundamentally contradicts what they believe, they usually ignore it as an anomaly. Those that have the courage to investigate it more often than not, because it goes against the grain of accepted and safe belief, face the ridicule and avoidance of their colleagues.