Mark "Monty" Montague (![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small} montyy0) wrote,@ 2007-11-11 09:59:00 |
|
peer review and scientific publications
A hoax paper denying global warming was put up on a fake journal site recently. A well-done scientific parody is worthy of entry into my hallowed blog halls as a nod to mischief, but this happens to tie in to quite a bit to an important topic that came up as I was writing about scientific publishing over the past few days.
Scientific publishing is a bit like a run-away train (or a system with large positive eigenvalues, since I know my audience): there are increasing number of peer-reviewed journals, because the publishers can make money selling them to universities, and because academics often benefit from quantity over (or in addition to) quality in their publication lists. As a result, there are so many scientific journals that even scientists in specific fields have trouble tracking them all, and certainly don't read them all. In addition to having a big impact on universities that have to pay for more journals that less people read (because there is always a possibility that an important article will show up in an obscure journal) this is likely leading to a wider range in the quality of what a "peer reviewed publication" means, because the most qualified peer reviewers in a field are unlikely to review for obscure journals. There are also games played by the publishers like requiring universities to get "bundled" journals so that their obscure journals get extra circulation even if they're not up to the standards that the university would normally subscribe to.
However, this hoax brings up a more direct problem, and a reason to applaud Congress for requiring that NIH-funded research be published open-access in PubMED Central. It has become all too common to see a reference to an article in some obscure journal, and to researchers frustrated that their university doesn't carry the article. Because journals demand that authors assign all publication and redistribution rights to the journal as part of accepting the article for peer review, often the journal disallows publication of PDFs on the authors' web sites and other forms of open publication. It is not unusual to find a journal reference, but have the site say "pay $30 for the full text of the article, unless your university has a subscription." It is also rather common to find that there is some sort of back-door, where starving students who can't afford $30 and can't talk their advisers into paying can download the article anyway (which, often, the publishers object to, but they don't appear to pursue as vigorously as music companies go after music pirates, possibly because there's some danger to the publishers should a court rule that these constitute "fair use.") The hoax site did an impressive imitation of an obscure peer-reviewed journal that offers articles as downloadable, and one of the many reasons it was believable is that the scientific publishing industry bloat has gotten to the point where it is routine for a researcher to run into a paper in an obscure journal related to his or her own field that is completely new to the researcher. It is assumed that if it quacks like a properly peer-reviewed journal, it must be a peer-reviewed journal.
In the case of the climate-change hoax article, the entire journal was fabricated and has no credibility at all. Unfortunately, the real danger is in grey areas. There are politically-motivated organizations, a "shadow scientific literature," so to speak, that produce journals that carry only articles that support specific public policy results. It has become the norm for "think tanks" to be groups of partisan "scientists" and pundits who crank out material supporting a political agenda. Proper peer review is the last line of defense against these, and it is in danger. Public access to certifiably peer-reviewed content is vital, as well, because when the validated scientific results are locked in the ivory towers of academia the public has no way to determine what is research accepted by an academic community versus what is synthetic pseudoscience in defense of some position.
In trying to write an article that started as an explanation of the NIH open-access requirement, I moved to one that raises alarm about the impending problems with scientific publishing having the ability to maintain its important contribution to the scientific endeavour, but even should those problems be resolved, there is a vital need for a real certification for properly peer reviewed articles that are actually representation of scientific consensus or scientific controversy, since there are many unscrupulous individuals who are willing to apply many of the same methods as the hoax author above, but with enough subtlety that they maintain credibility (and maybe even go through the motions of some sort of legitimacy and peer review, but it's "partisan niche" peer review.)
It is frequently raised as a defense of the scientific publishers' draconian intellectual property restrictions and high rates charged to universities that they are spending all of these profits on supporting the peer-review process. Even if this is true (although I find it worth pointing out that in many fields, peer reviewers work very hard and are not paid, so they tend to see graft when claims are made about the costs) I think it is a questionable connection. In fact, I see no unresolvable problems in the separation of the peer review process from the publication process entirely. If there were a system using some certification number, and something like PGP signatures for validation, to guarantee that a version of a paper had gone through a peer review process that was monitored by the professional societies or similar groups of known academic credentials, then researchers or their institutions could separate out the cost of peer review from the publication issues, and could retain all their rights once the peer review costs were met. All the research would be accessible to the public, which could validate whether it was "official" or not. Instead of having dubious indirect costs ("we need control the article in our copyright library to allow us to make a profit off of it to offset the cost we paid for peer review") whatever the costs of "herding cats" that gets volunteer or low-paid qualified scientists to review papers could be assessed directly, and charged appropriately to the authors, the research grants, the universities, or, perhaps, to publishers who want license to publish in paper journals.
A hoax paper denying global warming was put up on a fake journal site recently. A well-done scientific parody is worthy of entry into my hallowed blog halls as a nod to mischief, but this happens to tie in to quite a bit to an important topic that came up as I was writing about scientific publishing over the past few days.
Scientific publishing is a bit like a run-away train (or a system with large positive eigenvalues, since I know my audience): there are increasing number of peer-reviewed journals, because the publishers can make money selling them to universities, and because academics often benefit from quantity over (or in addition to) quality in their publication lists. As a result, there are so many scientific journals that even scientists in specific fields have trouble tracking them all, and certainly don't read them all. In addition to having a big impact on universities that have to pay for more journals that less people read (because there is always a possibility that an important article will show up in an obscure journal) this is likely leading to a wider range in the quality of what a "peer reviewed publication" means, because the most qualified peer reviewers in a field are unlikely to review for obscure journals. There are also games played by the publishers like requiring universities to get "bundled" journals so that their obscure journals get extra circulation even if they're not up to the standards that the university would normally subscribe to.
However, this hoax brings up a more direct problem, and a reason to applaud Congress for requiring that NIH-funded research be published open-access in PubMED Central. It has become all too common to see a reference to an article in some obscure journal, and to researchers frustrated that their university doesn't carry the article. Because journals demand that authors assign all publication and redistribution rights to the journal as part of accepting the article for peer review, often the journal disallows publication of PDFs on the authors' web sites and other forms of open publication. It is not unusual to find a journal reference, but have the site say "pay $30 for the full text of the article, unless your university has a subscription." It is also rather common to find that there is some sort of back-door, where starving students who can't afford $30 and can't talk their advisers into paying can download the article anyway (which, often, the publishers object to, but they don't appear to pursue as vigorously as music companies go after music pirates, possibly because there's some danger to the publishers should a court rule that these constitute "fair use.") The hoax site did an impressive imitation of an obscure peer-reviewed journal that offers articles as downloadable, and one of the many reasons it was believable is that the scientific publishing industry bloat has gotten to the point where it is routine for a researcher to run into a paper in an obscure journal related to his or her own field that is completely new to the researcher. It is assumed that if it quacks like a properly peer-reviewed journal, it must be a peer-reviewed journal.
In the case of the climate-change hoax article, the entire journal was fabricated and has no credibility at all. Unfortunately, the real danger is in grey areas. There are politically-motivated organizations, a "shadow scientific literature," so to speak, that produce journals that carry only articles that support specific public policy results. It has become the norm for "think tanks" to be groups of partisan "scientists" and pundits who crank out material supporting a political agenda. Proper peer review is the last line of defense against these, and it is in danger. Public access to certifiably peer-reviewed content is vital, as well, because when the validated scientific results are locked in the ivory towers of academia the public has no way to determine what is research accepted by an academic community versus what is synthetic pseudoscience in defense of some position.
In trying to write an article that started as an explanation of the NIH open-access requirement, I moved to one that raises alarm about the impending problems with scientific publishing having the ability to maintain its important contribution to the scientific endeavour, but even should those problems be resolved, there is a vital need for a real certification for properly peer reviewed articles that are actually representation of scientific consensus or scientific controversy, since there are many unscrupulous individuals who are willing to apply many of the same methods as the hoax author above, but with enough subtlety that they maintain credibility (and maybe even go through the motions of some sort of legitimacy and peer review, but it's "partisan niche" peer review.)
It is frequently raised as a defense of the scientific publishers' draconian intellectual property restrictions and high rates charged to universities that they are spending all of these profits on supporting the peer-review process. Even if this is true (although I find it worth pointing out that in many fields, peer reviewers work very hard and are not paid, so they tend to see graft when claims are made about the costs) I think it is a questionable connection. In fact, I see no unresolvable problems in the separation of the peer review process from the publication process entirely. If there were a system using some certification number, and something like PGP signatures for validation, to guarantee that a version of a paper had gone through a peer review process that was monitored by the professional societies or similar groups of known academic credentials, then researchers or their institutions could separate out the cost of peer review from the publication issues, and could retain all their rights once the peer review costs were met. All the research would be accessible to the public, which could validate whether it was "official" or not. Instead of having dubious indirect costs ("we need control the article in our copyright library to allow us to make a profit off of it to offset the cost we paid for peer review") whatever the costs of "herding cats" that gets volunteer or low-paid qualified scientists to review papers could be assessed directly, and charged appropriately to the authors, the research grants, the universities, or, perhaps, to publishers who want license to publish in paper journals.
