I am quite puzzled by the first piece. The authors say very little about their methodology: “As a starting point we used citation counts and numbers of searches in JSTOR….only to give us a large group of eligibles,” they state. They briefly touch upon the inherent bias arising from using citation count, then claim that they had appealed to their own judgment to fashion their final list, which is displayed in alphabetical order. The list is subtantiated with abstracts explaining why each chosen paper is “admirable” – it led to “many applications,” it “anticipated” core issues that occupies economists today, it has become “standard” for the analysis of such and such economic feature, it resolved a long standing debate, it launched a field. There is thus no sense of what the initial data set was and how the authors transformed it to achieve their top 20.
I treated the paper as a historical object, one bearing clues of how economists identify and construct their sources of inspiration, lineage, and standards of great work. Yet, reconstructing the initial set of data hadn’t proved easy. I found three lists that the authors may have used : one covering the whole range of AER issues and using “citerank”as a classification device. (“citerank” is not equivalent to “number of quotations” nor yield the same ranking. However, I’ve been unable to find out how the “citerank” algorithm work. I may have missed the relevant explanation webpage, but shouldn’t such information be readily available when making a query)
Papers can also be ranked beginning with the ” most cited in the past three years” or with the “most accessed in the past three years.” These rankings are updated every month, so that it’s impossible to access those used by the authors.
I was surprised to see how much the AER committee remained close from the original rankings. Of the 20 final papers, 14 are taken from the top 20 citerank ( Alchian-Demetz (72) on information costs and economic organization, Arrow (63) on uncertainty and medicare, Deaton-Mullbauer (80) on the demand system, Diamond (65) on national debt, Diamond-Mirrlees (71) on optimat taxation, Dixit-Stiglitz (77) on monopolistic competition and optimum product diversity, Friedman’s 1968 presidential address on monetary policy, Harris-Todaro (70) on migration, unemployment and development, Hayek (45) on information and knowledge, Krueger (74) on rent seeking behaviors, Krugman (80) on trade and product differentiation, Kuznets (55) on growth and income inequality, Lucas (73) on output-inflation tradeoff, Modigliani Miller (58) on the cost of capital and investment decisions, Mundell (61) on optimum currency areas), and 11 of them appear in another of the rankings abovementionned.
Of the remaining articles, Shiller (81) on stock prices can be found among the papers most cited in the past 3 years, Cobb Douglas (28) on the production function was evidently chosen to offset the bias toward moderately recent papers. I found no trace of Jorgenson (63) on capital theory , Ross (73) on the principal agent theorem and Grossman Stiglitz (80) on the impossibility of informationally efficient market.
More meaningful perhaps are the papers eligible and yet not selected for the top 20.
For instance, Jensen (86) on agency costs appears prominently in all three rankings, as well as Stiglitz and Weiss( 81) on credit rationing. Schultz (61) on investment in human capital, Shapiro and Stiglitz (84) on equilibrium unemployment as a worker device, and Stiglitz and Weiss (81) on credit rationing on markets with imperfect competition also show up in the top of two rankings each. Other papers scoring once include the D e Gustibus by Stigler and Becker (1977), Chenery (1960) on industrial growth, Baumol (1967) on unbalanced growth, Patinkin (1948) on Price flexibility and unemployment, Tobin (1972) on inflation and unemployment, Simon (1979) on rational decision making in business organizations, Williamson and Leibenstein (66) on efficiency X, and various pieces by Kahneman or Bernanke.
The authors’ heavy reliance on the JSTOR lists raises questions about the interpretation of such automatically generated rankings and more generally about the kind of qualitative conclusions one can reach through bibliometric analysis. Aside from my difficulties in identifying how the citerank ranking is constructed, I find myself loosely toying with citation and referencing practices and their sociological implications, with how genealogies and canons are fashioned by the profession, and, given the peculiar status of the AER within the discipline, with the idea of a “construction of a mainstream,” whatever the dangerous word ‘mainstream’ means.
Together with Clement Levallois, we then played the ranking game with hope of generating some objets we could compare to the AER top 20.
We looked at JSTOR overall ranking(including all discipline and by citerank). Two articles written by scientists usually considered as economists surfaced in the top 10. Rank 6, Coase and the Problem of Social Cost (1960) published in Law and Economics. Rank 7, Becker and the Theory of the allocation of time (1965) published in the Economic Journal. Neither authors to be found in the AER top 20.
The top articles (citeranks) in social sciences also brought some results as regards economics. Rank 1 is Coase’s problem of social cost, rank 2 is Becker’s 1965 paper on the allocation of time, and rank 3 is Coase again on the nature of the firm (1937), published in Economica. When it turned out that the top cited articles in economics (which I assumed was a subpart of the social science ranking) involved neither Becker nor Coase’s paper, I got lost and ended the game.
Neither this AER top 20 nor the second anniversary paper, an introductory piece to the history of the journal which relied on quantitative data to account for the evolutions in number of pages, volumes, funding, content repartition… claim to meet some history research standards. “This is not a history-of-thought paper—rather, it is an economic history paper about a central institution of the economics profession,” Robert Margo points out. Yet, the choices of the AER editorial board for this special issue have proved disturbing to historians (see for instance t
hat comment by Roy Weintraub).
My own disappointment is not with these papers, but rather with the limited editorial ambition they embody. We’re told that there’s an anniversary to celebrate, and we end up with two papers lost in a regular issue. There could have been so much more: papers where contributors’ background (institution/ PhD institution) would be mapped and compared to the ecology of other journal (QJE, JPE, econ Journal, Econometrica etc…) thanks to network analysis conducted on JSTOR data.; a study of the evolution of the subjects handled, one that would be tied to the emergence of new disciplinary subfields and the waning of others. An analysis of how the referee process worked and evolved, and of the consequences of such patterns for the profession, one based on archive analysis (the
AER archives, located at the Duke paper project still awaits exploration. Editorship styles, visions, and impact, could have been assessed with the help of oral history. All these investigations would have help the reader understand the intellectual and institutional role of the AER within the profession. I wish the reason for the editors’ timidity is that they see such a volume as plain scifi. I fear they’d just consider it worthless. My own disappointment is not with these papers, but rather with the limited editorial ambition they embody. We’re told that there’s an anniversary to celebrate, and we end up with two papers lost in a regular issue. There could have been so much more: papers where contributors’ background (institution/ PhD institution) would be mapped and compared to the ecology of other journal (QJE, JPE, econ Journal, Econometrica etc…) thanks to network analysis conducted on JSTOR data.; a study of the evolution of the subjects handled, one that would be tied to the emergence of new disciplinary subfields and the waning of others. An analysis of how the referee process worked and evolved, and of the consequences of such patterns for the profession, one based on archive analysis (the AER archives, located at the Duke paper project still awaits exploration. Editorship styles, visions, and impact, could have been assessed with the help of oral history. All these investigations would have help the reader understand the intellectual and institutional role of the AER within the profession. I wish the reason for the editors’ timidity is that they see such a volume as plain scifi. I fear they’d just consider it worthless.
