Trying to assess the importance of particular scientific papers has long been a tricky task. The current system relies on counting the number of times a paper is cited by others to determine how large an effect it has had on subsequent research, but this number can be misleading, a new study notes. Simply
counting citations favors disciplines such as biology, where papers tend to be cited more, over fields such as mathematics, where citations are less frequent. In addition, a citation from a relatively marginal paper counts just the same as a citation from a leading researcher publishing in a marquee journal [Scientific American].
To try to get around these problems, a pair of researchers decided on a different tactic: They took the algorithm that Google uses to determine how to rank the Web pages turned up in a search result, and used it to rank the importance of scientific articles. The Google PageRank algorithm checks the number of times each Web page is linked to in order to determine its importance, which is equivalent to counting citations. But it has several other aspects that were very useful when applied to ranking scientific papers. The algorithm gives greater weight to citations from papers that list only a few references, and also to citations from papers that are themselves often cited.
"Because of these attributes, PageRank readily identifies a large number of scientific 'gems'--modestly cited articles that contain ground-breaking results" [arXiv]
, the researchers write. Among those gems turned up in the researchers first experiment were nine papers written by future Nobel Prize winners. In an experiment described in the Journal of Neuroscience [subscription required, but the paper is available for free on arXiv], the researchers tested their theory by applying Google's PageRank algorithm to over 350,000 articles published in the American Physical Society's family of journals. They looked at citations between articles within that set, and found that for the most part, the PageRank system and rankings based on citation numbers identified the same papers as being scientifically important. But there was a twist.
Interestingly, they found a set of outliers where the PageRank number was substantially higher than would be expected based on citation numbers alone. You may recognize some of the authors: Bardeen, Cooper, and Schrieffer; Weigner and Seitz; Onsager; Kohn and Sham; Feynman and Gell-Mann. In fact, nine of the ten outliers were Nobel Prize-winning papers and the tenth, Cabbibo's "Unitary Symmetry and Leptonic Decays," is arguably the largest omission in Nobel history [Ars Technica].
While the researchers conclude by arguing that statistics are no match for scientific judgment when it comes to assessing the quality of research, no doubt some curious souls are already applying the algorithm to more recent sets of papers in attempt to sniff out the next Nobel Prize winner. Related Content: 80beats: Work With Fluorescent Jellyfish Protein Wins Nobel Prize in Chemistry 80beats: Nobel Prize in Physics Awarded to Particle Physicists 80beats: Nobel Prize for Medicine Awarded to Virus HuntersImage: Wikipedia
