INET News

August 24, 2010

Front Page of the NYT: Scholars Test Web Alternative to Peer Review

The New York Times has now pushed to the front page of today's paper an issue of real relevance to INET: how new web tools are beginning to upend traditional peer review in academia. 

"Scholars Test Web Alternative to Peer Review" shows the early signs of how the new tools and the younger champions of them are starting to challenge the traditions that have been around for decades if not centuries. Check out a few quick excerpts from the Times:   

Today a small vanguard of digitally adept scholars is rethinking how knowledge is understood and judged by inviting online readers to comment on books in progress, compiling journals from blog posts and sometimes successfully petitioning their universities to grant promotions and tenure on the basis of non-peer-reviewed projects....

Just a few years ago these sorts of developments would have been unthinkable, said Dan Cohen, director of the Center for History and New Media at George Mason University. “Serious scholars are asking whether the institutions of the academy — as they have existed for decades, even centuries — aren’t becoming obsolete,” he said....

To Mr. Cohen, the most pressing intellectual issue in the next decade is this tension between the insular, specialized world of expert scholarship and the open and free-wheeling exchange of information on the Web. “And academia,” he said, “is caught in the middle.”

INET has seen this development early, and is promoting this new thinking, and the new, younger thinkers who often champion these ideas. We also are introducing new tools and approaches as we begin to build a community around this website.

I always consider an idea that makes the front page of the New York Times as the sign that the idea is now getting mainstreamed or legitimatized. And so here's one more sign that big changes are coming to Academia sooner than most think....

Peter Leyden

Director, INET Online

Posted by Peter Leyden at 5:10 pm

Comments

Submitted by Paul Cottrell on Tue, 08/24/2010 - 9:04pm.
0

The NYT article is a great cursory review of a new shift in education and in what students want. Through technology, students can collaborate in ways our professors could only dream of during their tenures. Peer reviewed scholarship is critical and very important to building knowledge and establishing "truth" in a particular field, granted that "truth" may change with new research and perspectives.

The point that I would like to make in this essay is quite simple, peer review is important, but so is socialization of ideas. Through technology we can share ideas and question long held beliefs in a more free flowing way. This collaboration can build good friendships and promote networking in a researcher's field of study. Education or learning should not be centralized due to the modern constructs we have available, i.e. social technology, online libraries, and modeling software.

The world of education has changed, and that new paradigm is what is called a scholarly-practitioner; where the scholar is applying their profession to promote social change, hopefully in a positive way. Research for the sake of research maybe fun for some but usually does not promote social change. It is through collaboration with the wider establishment where ideas can percolate.

In conclusion, peer review research is essential and should remain, but social connectivity and interactivity will increase the learning experience and promote scholarship and application. May we all find a plot of land on this new continent we call knowledge.

Paul Cottrell, MBA
Walden University
paul.cottrell@waldenu.edu

Submitted by Peter Leyden on Wed, 08/25/2010 - 8:28pm.
0

 I like your distinction of the scholarly-practitioner. There is a great need to get academics more connected to what is happening in the real world, and moving their work more into the open web is one way to do that. 

Rob Johnson, INET's executive director, has a nice way of putting it in our short video trailer that is the first video in the player off the front page. To paraphrase, he said it's time to set the social goals first and get the economy aligned with those goals and more generally working in service to society.  

Bridging the gap between Academia and people outside of Academia through new media is one way to help do that. 

Submitted by DanielDWilliam on Tue, 08/31/2010 - 8:51am.
0
My response to the NYT article
 
 
and some comments in Ben Goldacre’s Bad Science website about the difficulties of academics engaging with the public and young academics struggling to over come editorial bias.
 
http://www.badscience.net/
 
I would propose a semi-formal wiki-journal. (Paid for out of on-line advertising revenue). This would cover all aspects of research. I suggest that the only reason why specialist journals exist is to make searches for relevant content efficient in a world before searchable indexes and #hashtags.
 
Anyone could publish a paper and anyone could comment.
 
You might consider having an editorial board screening the postings to weed out utter dross and insanity; or screen the first few submissions by individuals but I think there is no need for a quality gate-keeping role. Public exposure should be enough of a quality screen.
 
Comments are then invited. Most of the debate would happen in the comments area.
 
For those looking for high quality papers I would propose a simple quality grading system.
 
Those commenting should review the paper, perhaps using something as simple as a star rating, such as Amazon uses, or a more complex and targeted scoring system to rate the quality of the article. ( For example, scores could be awarded for methodological rigour, correct application of appropriate methodology, conclusions supported by data, )
 
Comments on the quality of the paper may lead to a revision of the article by the authors.
 
In turn, those reviewing papers are themselves reviewed. Each would be assessed, again either by an editorial review board or by the whole community based on the perceived correctness of their scoring of papers
 
This double feedback loop would allow an uninformed audience to judge
 
As a summary the paper has a simple quality stamp that allows readers quickly to assess how good an article it is. Details should be available to support the overall score, allowing transparency of the quality assessment.
 
The build of actual comments, links or citations, and the quality scores; adjusted for the quality of the scorer should harness new technology to disseminate new work and quickly challenge, edit, amend, improve or debunk them. Scientists could collaborate quickly.  Poor quality science would be quickly and openly challenged. Science, properly identified as high-quality science, would be more available to the public.
 
This system would allow more open access to academic papers, without editorial bias or publication delay. Scientists could collaborate across disciplines more easily and the public would have ready access to the raw stuff of science.
 
One additional benefit would be that harassed journalists short of time would be quickly able to see which criticism of a particular article or view point had the backing of considered opinion. This should improve the quality of science journalism in the main stream media.
Submitted by Paul Cottrell on Wed, 09/01/2010 - 6:11pm.
0

Daniel,
You have an interesting concept of a rating system, but scholarly journals should remain to help promote their respective fields of study. You had an interesting response on how to couple peer review and public review.

Thanks,
Paul Cottrell
Walden University
paul.cottrell@waldenu.edu

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <blockquote> <br> <cite> <code> <dd> <div> <dl> <dt> <em> <h2> <h3> <h4> <img> <li> <ol> <p> <span> <strong> <sub> <sup> <table> <tbody> <td> <tr> <ul>
    Allowed Style properties: display, float, height, margin, margin-bottom, margin-left, margin-right, margin-top, width
  • You may insert videos using embed codes like these:
    • [video_large:KoqLu5CKx-o]
    • [popupvideo_mini:KoqLu5CKx-o right]
    • [lightboxvideo_mini:KoqLu5CKx-o]
    • [text_popupvideo:KoqLu5CKx-o nostart noicon|Click here to open the video.]
    • [text_lightboxvideo:KoqLu5CKx-o|Open this video in a lightbox.]
    To learn more, please click on the "More information..." link below.

More information about formatting options