Cosma Shalizi - Why Economics Needs Data Mining

About the Interview

Cosma Shalizi urges economists to stop doing what they are doing: Fitting large complex models to a small set of highly correlated time series data. Once you add enough variables, parameters, bells and whistles, your model can fit past data very well, and yet fail miserably in the future. Shalizi tells us how to separate the wheat from the chaff, how to compensate for overfitting and prevent models from memorizing noise. He introduces techniques from data mining and machine learning to economics -- this is new economic thinking.

About Cosma Shalizi

Cosma is an assistant professor of statistics at Carnegie Mellon University, where his research focuses on aspects of the statistical analysis of complex systems: nonlinear prediction algorithms, heavy-tailed distributions, contagion in networks, and self-organizing processes. He got his Ph.D. in theoretical physics from the University of Wisconsin-Madison in 2001. Full profile



Professor Shalizi is an incredibly interesting man. I had no clue his interests included economics.


Excellent discussion. Dr. Shalizi's critical approach to model building and uses in Economics was long overdue. His superb background will allow him to give empirical content to what many economists have been complaining for decades. I believe thatt there is a very good chance for Economics to become more pertinent and sound for policymaking with his proposal. Congratulations Dr. Shalizi. I see you with a Nobel in the future.

Economist from Puerto Rico.


Judging only by its selection of Professor Shalizi I can tell INET is going to be changing the world for the better.


I don't think Shalizi and Mehrling are ever in the same frame. Are we sure they aren't the same person? I swear a very good make up artist could be behind this...

In all seriousness, great interview.


I seriously admire this new Economic thinking. As a young Economic lecturer in a developing country University, I seriously feel that our perception about modelling need to be changed. Of late, students and supervisors in some schools especially in Macroeconomics believes that a model must be so complex for it to be good, the size of data available not withstanding.

I believe INET is here for a more policy oriented economists


Thanks! Very clear explanation! I would love to see/hear/be involved in conversations specifically about the types of data that might be useful to validate or disprove models, or to facilitate mining or other techniques.


Interesting, but certainly not relevant to all areas of economics--most economists should not be urged to "stop doing what they're doing." (I get frustrated when non-economists who, while they may have something very useful to say about particular methodological practices, suggest their critique applies to all areas of the field.)

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <blockquote> <br> <cite> <code> <dd> <div> <dl> <dt> <em> <h2> <h3> <h4> <img> <li> <ol> <p> <span> <strong> <sub> <sup> <table> <tbody> <td> <tr> <ul>
    Allowed Style properties: display, float, height, margin, margin-bottom, margin-left, margin-right, margin-top, width
  • You may insert videos using embed codes like these:
    • [video_large:KoqLu5CKx-o]
    • [popupvideo_mini:KoqLu5CKx-o right]
    • [lightboxvideo_mini:KoqLu5CKx-o]
    • [text_popupvideo:KoqLu5CKx-o nostart noicon|Click here to open the video.]
    • [text_lightboxvideo:KoqLu5CKx-o|Open this video in a lightbox.]
    To learn more, please click on the "More information..." link below.

More information about formatting options