Keeping Score: CJR Presents a New Paradigm for Rating Journals

Article ImageJournal rankings: What is it about this modest little metric that causes such uproar in the research community?

Hiring, tenure, and grant awards are often influenced by whether or not one's research has been published in a journal deemed "prestigious" by its placement in the ranks. Libraries routinely use rankings to justify or slash subscriptions. More troubling is the accusation that the direction of research has been skewed toward the publication of "fashionable" or high-profile topics, the kind that get into the journals with the highest rankings, while less popular but important work gets passed over. For decades, the gold standard for journal ranking has been ISI's Impact Factor (IF), which is determined by a simple yet elegant algorithm: Tally the number of citations to a journal, and divide this by the number of articles published in that journal over a two-year period.

While there has been much clamor over using quantitative measures to assess the quality of a journal, subjective, qualitative measures like surveys have fared no better. There may be agreement among experts about the top three or four journals in a field, but the evaluations become murkier concerning the smaller or more specialized journals down the list.

Enter the Center for Journal Ranking (CJR), a web service launched in January by Andrew Lim, associate professor at the Hong Kong University of Science and Technology and founder of Red Jasper, a high-tech start up, specializing in applying logistics to business, transportation, and supply chain management. Lim's aspiration is "…to be able to affect and change the ways of the academic research world in a positive way, where we can safely eliminate the subjectivity in [rankings] in as many ways as possible, and yet be able to back up the rankings in as objective and meaningful a way as possible."

Consequently, the Center for Journal Ranking in its infancy has adapted the PageRank methodology (made famous by Google) to rank thousands of journals "in a multitude of disciplines." While Google uses PageRank to rate a page by how often it is requested, CJR extends its capabilities by using complex algorithms to differentiate between the quality or significance of the citing source.

CJR proposes two new indicators to augment the PageRanking protocol: the Journal Influence Index and the Paper Influence Index, measures that are then compared with rankings derived from expert surveys. "Our goal," says Lim, is to enable scientific proofs to trump—or better yet, to validate—subjectivity." CJR aspires to be the definitive authority in journal ranking. It experienced over 400,000 hits in its first month, and it has established a forum as platform for discussion and broad-based consultation with researchers in order to continually refine the work in journal ranking.

Registration to CJR and access to their rankings for over 7,000 journals is free. "It's too early to ponder a business model," says Lim. "At this moment, we are only concerned about what kinds of impact the Center has to bring to academics and researchers around the world. I should say that we are just at the stage of validating our contribution."