The Institute Blog

Hogarth and Soyer on the Hollow Men

We are the hollow men
We are the stuffed men
Leaning together
Headpiece filled with straw. Alas!
Our dried voices, when
We whisper together
Are quiet and meaningless
As wind in dry grass
Or rats' feet over broken glass
In our dry cellar
Shape without form, shade without colour,
Paralysed force, gesture without motion;
Those who have crossed
With direct eyes, to death's other Kingdom
Remember us-if at all-not as lost
Violent souls, but only
As the hollow men
The stuffed men

- - - -

Justin Fox in his Harvard Business Review blog brings to our attention the dangers of false certainty and how representation of data can contribute to, or dispel the false certainty of statistical and econometric technique.

He highlights the research of Emre Soyer and Robin Hogarth and particularly their paper entitled “The Illusion of Predictability: How Regression Statistics Mislead Experts.” (The paper is attached at end of this post.)

"My concern," Hogarth recently said while presenting the results, "is that when reading economics journal articles you get the impression that the world is much more predictable than it is."

Fox highlights some of the remedies for fighting the illusion of false precision, including the Bank of England’s use of “Fan Charts” and other visual representations of data.

Selling false certainty would seemingly be a limited market. Bad guidance would be weeded out by the false results, as the story goes. I am not so sure. The demand for relief from anxiety breeds an oversupply of alleged authority. The product being sold is not insight or truth. The product being sold is a service that helps you feel better when you are nervous. Snake oil salesman, according to George Akerlof and Robert Shiller, in their book Animal Spirits, can fill this void temporarily. And then they are despised when their story is shown to be false. But the yearning remains and demagoguery of various forms rushes to fill the void.

Riding behind the shield of “science” rather than engaging in other forms of rhetorical persuasion is one way to appear certain. Intimidation through arcane methodology is another way to remove the debate. Taking people into dark lacunae of technique and ridiculing them or bamboozling them with this or that complex representation of a simple argument often leads to deference. We have all seen these rituals in action. They are the stock in trade of what Robin Hogarth calls, “The Arrogant People,” a term he fastens with particular focus to the economics profession.

Or as the blogger Borepatch reminds us:

1. "If anything else, please remember this when someone says ‘The science is settled.’ Unambiguous settled science usually means someone wants to convince you do something you don't want to do, and they've done more research than you."

2. "Remember, the Scientific Method isn't replacing a Falsehood with a Truth. It's replacing a Falsehood with a more subtle Falsehood.”

I have nothing against technique in service of illumination. But science and formalism are not the same. The key word here is method. Method is no substitute for insight and is not sufficient to persuade.

-INET Executive Director Robert Johnson

Click here to read Fox's post

Click the attachment below to download Hogarth and Soyer's study

Soyer & Hogarth_2012.pdf1.78 MB



Thanks Robert for this blog post and the links - sent from the student desk, writing a method chapter.

Related to the issues that Fox and Soyer & Hogarth take up, I would like to bring your attention to book:

The Cult of Statistical Significance - How the standard error cost us jobs, justice and lives, from 2008 by Deirdre N. McCloskey and Stephen T. Ziliak

Regards, Asgeir


John Maynard Keynes made the point about the limited applicability of the standard normal distribution outside the natural sciences in the Treatise on Probability and Benoit Mandelbrot later confirmed this with regard to econometric techniques and finance (according to Dr. Michael Emmett Brady, at least). Though this isn't exactly new, it's good to see public awareness on this issue increasing.


Sometimes "New" is not synonymous with novelty. Rather what is new is a change in the consensus of how we perceive the world and act in light of recent experience. What is then new is a change from how we perceived relative to that past.

I believe that flawed views of false certainty in financial expectations have lead to very significant flaws in the design of financial products and financial regulatory systems. If fundamental radical uncertainty is taken seriously, building on the vision of Keynes, Knight, Hayek, Minsky and Soros, we will likely see a very different perspective on financial system architecture that supports our society. These ideas have been around for a awhile. In that respect you are correct that it is not a new vision.

Acting as if this was a truer vision than using certainty equivalents as a proxy for fundamental uncertainty would lead to a very profound change in behavior and social convention. That would in my view be "new" and healthy.


Is that the real Dr. Robert Johnson himself? If so, thanks for your response! I didn't expect a response from you myself.

I agree with you that following and acting upon the tradition of Keynes and Knight would be "new and healthy", Dr. Johnson.

But perhaps a better term than "radical uncertainty" would be a term used by Robin Hogarth in one of his articles before, and a term used by the fabled Daniel Ellsberg: "ambiguity".

This ambiguity can be explained in decision theoretic terms via John Maynard Keynes's A Treatise on Probability itself. I highly recommend all INET scholars to take a look at the work of Dr. Michael Emmett Brady (see the links below). His research on Keynes's TP is important to explaining uncertainty/ambiguity and has potential policy implications...


A future economic system must be able to deal with uncertainty in powerful dynamic, and adaptable ways. My own evolving work on Transfinancial Economics represents a real breakthrough..yet to be fully recognized.However, some economists (if they can understand it)have found it of interest.

The term Trans in Transfinancial implies above, or beyond conventional mainstream finance. In this context, it does not imply ofcourse money being sent from one country to another. The use of the term Economics indicates its wider dimension beyond finance.


In order to achieve what you, Mr. Johnson, propose justifiably, we have to begin as early as possible with regard to economic education in the universities around the world.
When studying economics, the only thing you are always told as being crucial is to do as much math as possible. In Germany, for example, I will have to spend about 80% of my time doing statistics, analysis etc. Yet, all this math stuff is mainly gearded to enabling economists to make predictions about future developments of any kind. Thus, economics negatively changes from a social science to a kind of technique for predictions. But economics is more than just predictions about future. It is about fundamental human interactions, about history, about morality.
By letting students do only math we focus economics on one tiny little aspect (which is important but not transcendent!)and lose so much more.
In fact, economics lives of uncertainty. As a social science (we must remember this fact, it is not an objective natural science)it lives of different and subjective views. In short, if there was no uncertainty about future, we wouldn't need true economics any longer.
So the most healthy thing we can do to revalue true economics, thus helping build better social conventions is to change the way of teaching economics and greatly reduce mathematical contents.


I could not agree with you more.

First of all, history and institutional context matter. Universal laws appear to me to have very little applicability in economics. Using models as simplifying technique does not mean one should then devalue information about the economy in favor of information about modeling technique. When I was in school one had the choice of going into operations research rather than economics if modeling technique was the forte.

Secondly, ideas are formed/envisioned in context. History of Economic Thought where the context, social structure, vested interests, and economic structure are understood alongside the ideas themselves would help us all to see that these are not timeless universally valid ideas in many cases. It kind of breeds humility.

Understanding the correspondence between the presuppositions in a model and the underlying society being modeled is important to assessing the VALUE of that model. As I argue in this post, presupposing you can bolt down the future when you cannot is a very significant error where modeling techniques that are tractable are imposed upon vision of how society works with potentially significant costs. What I call "bad maps"

False certainty has a lot of attractions and it requires a deep and profound education to see through it. Few, when faced with life's practicalities and ambitions make a stand against it adopting these customs.



We are Emre and Robin, the authors of the study featured in this post.

First of all, we would like thank Robert Johnson for his insightful discussion of the study and all the commentators for their remarks.

This study is now a discussion paper in International Journal of Forecasting. Hence, we would like to contribute to the ongoing discussion by posting links to the comments made on the study and our reply to those comments.

Due to copyright issues, we cannot share freely the journal versions, but can put links to the last working papers.

Here is the study:

Here is the comment paper by Scott Armstrong:

Here is the comment paper by Stephen Ziliak:

Here is the comment paper by Nassim Taleb and Daniel Goldstein:

Here is the comment paper by Keith Ord:

Here is our reply to the comments:

Thank you once more for the engaging discussion.

Best wishes,

Emre and Robin

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <blockquote> <br> <cite> <code> <dd> <div> <dl> <dt> <em> <h2> <h3> <h4> <img> <li> <ol> <p> <span> <strong> <sub> <sup> <table> <tbody> <td> <tr> <ul>
    Allowed Style properties: display, float, height, margin, margin-bottom, margin-left, margin-right, margin-top, width
  • You may insert videos using embed codes like these:
    • [video_large:KoqLu5CKx-o]
    • [popupvideo_mini:KoqLu5CKx-o right]
    • [lightboxvideo_mini:KoqLu5CKx-o]
    • [text_popupvideo:KoqLu5CKx-o nostart noicon|Click here to open the video.]
    • [text_lightboxvideo:KoqLu5CKx-o|Open this video in a lightbox.]
    To learn more, please click on the "More information..." link below.

More information about formatting options