Non-routine Change, Imperfect Knowledge and Economic Science
1. The Limits to Knowability
We are reminded time and again that our understanding of the process driving market outcomes is at best incomplete. Yet the post-1945 history of macroeconomics and finance theory attests to our reluctance to acknowledge that there are inherent limits to what we can know about how outcomes unfold over time. Indeed, over the last four decades, economists have come to believe that, in order to be worthy of scientific status, economic models must generate “sharp” predictions that specify exactly all potential changes in outcomes and the probabilities with which they might occur – in the past, present, and future all at once.
Paradoxically, this conception of economic science – that we can discover such exact knowledge – has made it more difficult for us to understand real-world markets and the economies in which they are embedded, for it ignores why market participants and economists confront inherent limits to knowability. Simply put, change in capitalist economies is to a significant degree non-routine, and thus cannot be adequately represented in advance with mechanical rules and procedures. In modern economies, individuals and companies engage in innovative activities, discovering novel ways to use existing physical and human capital, as well as new technologies in which to invest. Market participants, meanwhile, search for and occasionally adopt new ways to forecast returns from their activities. And the institutional and broader social context within which these activities take place also changes in novel ways.
The history of economic thought includes widely differing approaches to the daunting analytical challenge that non-routine change poses. The largely narrative mode of analysis used by early modern economists – Friedrich Hayek, Frank Knight, John Maynard Keynes, and their contemporaries – enabled them to examine the opaque interdependence between market outcomes, forecasting by profit-seeking participants, and other features of the social context, including economic policies and institutions.
By placing non-routine change and the imperfect knowledge that it engenders at the center of economic analysis, the giants of early modern economics arrived at remarkably powerful and durable insights. Despite their profound differences concerning the role of markets and the appropriate scope of public policy, they all recognized that purposeful behavior, whether motivated by pure self-interest or other objectives, cannot be completely comprehended, in terms of some set of causal factors, by outsiders, be they economists, policy officials, or social planners. Consequently, market outcomes that result from the decisions of many individuals are not completely intelligible, either.
Knight’s arguments concerning the importance of “radical uncertainty” led him to question the relevance of standard probability theory for understanding business decisions. He argued that standard probabilistic portrayals of decisions cannot adequately characterize how profit-seeking individuals respond to change, or how market outcomes unfold over time.
Keynes shared Knight’s profound doubts about the usefulness of standard probability theory for understanding change in individual decision-making and market outcomes. As he put it, we “cannot depend on strict mathematical expectation, since the basis for making such calculations does not exist.” The importance that Keynes attached to the role of uncertainty concerning both outcomes and probabilities played a key role in his analysis of financial markets and their influence on the broader economy, particularly investment.
Likewise, Hayek argued that “the economic problem of society is a problem of the utilization of knowledge which is not given to anyone [including economists] in its totality.” Hayek’s dictum implies that no mathematical model can fully mimic what markets do, and this observation led him, in his Nobel lecture, to expose the scientific pretense of those modes of economic analysis that purport to account for individual decision-making and market outcomes with models that assume away imperfect knowledge. As he put it, “I confess that I prefer true but imperfect knowledge…to a pretense of exact knowledge that is likely to be false.”
In contrast to these skeptical views, the vast majority of contemporary economists have been much less circumspect about the ability of economic analysis to uncover exact probabilistic representations of the causal processes that underpin change in capitalist economies. Indeed, an overwhelming majority of contemporary macroeconomic and finance models imply such overarching representations.
In order to construct such models, which Roman Frydman and Michael Goldberg refer to as fully predetermined, economists must assume away the importance of non-routine change. These models presume that an economist can fully pre-specify, in terms of some set of causal factors, how individuals make decisions, and how the resulting market outcomes unfold at all points in time. Such overarching accounts of individual behavior and aggregate outcomes represent change as a mere random deviation from a fully predetermined path. As a result, the currently prevailing conception of economic science presupposes that, in principle, there are no limits to what economists can know about change.
Assuming away non-routine change does not, however, eliminate its importance for understanding outcomes in capitalist economies. This is particularly apparent in financial markets, whose participants revise their forecasting strategies at moments and in ways that they themselves, let alone economists and other outsiders, cannot fully foresee. Because these non-routine revisions alter how market outcomes unfold over time, sooner or later, any overarching model of these outcomes becomes inadequate.
That is why contemporary macroeconomic and finance models have repeatedly been found to be grossly inconsistent with time-series data. Frydman and Goldberg argue that these models also suffer from irreparable epistemological flaws, and they trace their empirical and theoretical difficulties to the core premise that fully predetermined accounts of change are possible.
Contemporary economists’ quest for a model that could predict the complete set of future market outcomes in probabilistic terms is not the first such endeavor in the social sciences. In his seminal refutation of the claim that “historicism” might one day enable social science to “predict the future course of history,” Karl Popper pointed out that any such approach is futile “to the extent to which [historical developments] may be influenced by the growth of our knowledge.”
Although Popper was strongly critical of attempts to develop fully predetermined accounts of history, he was quick to point out that his
argument does not, of course, refute the possibility of every kind of social prediction; on the contrary, it is perfectly compatible with the possibility of testing social theories – for example, economic theories – by way of predicting that certain developments will take place under certain conditions.
2. Imperfect Knowledge Economics
Imperfect Knowledge Economics builds on Popper’s insight that, though contingent on how knowledge unfolds, economic prediction that can be confronted with empirical evidence is possible. Its mathematical models explore the possibility that change and its consequences can be portrayed with qualitative and context-dependent conditions. These conditions are also contingent insofar as the qualitative regularities that they formalize become manifest – or cease to be relevant – at moments that no one can fully foresee.
Frydman and Goldberg show how opening extant macroeconomics and finance models to non-routine revisions of participants’ forecasting strategies helps us to overcome their epistemological flaws while affirming the possibility of formal macroeconomic and finance theory. Indeed, Frydman and Goldberg portray how asset prices and risk unfold over time with a mathematical model. The qualitative and contingent predictions generated by an IKE model of asset-price swings exemplify what Popper would regard as a feasible goal of economic theory. Although the model predicts that, under “certain conditions,” an asset price will undergo a sustained movement in one direction, it does not predict when such upswings or downswings will begin or end.
Despite its reliance on qualitative and contingent conditions, an IKE model of swings in asset prices and risk can be rigorously confronted with empirical evidence, just as Popper suggested. In a recent paper, Frydman, Goldberg, Søren Johansen, and Katarina Juselius show that this model provides a significantly better account of such swings than its fully predetermined and supposedly “rational” counterpart.
Frydman and Goldberg also argue that recognizing the limits to knowability leads to a new way of thinking about the balance between financial markets and the state in modern economies. This rethinking arises from their IKE model’s explanation of why asset-price swings sometimes become excessive, and how the hitherto neglected relationship between inherent asset-price instability and financial risk helps us to understand how excessive swings come to an end. This analysis provides a conceptual framework for macroprudential policy aimed at dampening excessive price swings, and thus reducing the social costs inflicted when they reverse direction.
These findings suggest that recognizing the inescapable limits to our knowledge that arise from non-routine change may be the key to bringing macroeconomic and finance models closer to reality, thereby enhancing their relevance to policymaking and, more broadly, to public discussion about the relative roles of financial markets and collective action in modern economies.
3. Integrating Fundamental Factors with Psychological and Social Considerations
The search for fully predetermined models that accord an explicit role to market participants’ forecasting behavior in driving outcomes, such as asset prices, led economists and finance theorists to two classes of models: so-called rational expectations models that focus on the role of fundamental factors, and behavioral-finance models that emphasize non-fundamental factors, such as psychological and social considerations. But this dualism of fundamental and non-fundamental factors is largely an artifact of both classes’ fully predetermined structure. In fact, economic fundamentals and non-fundamental factors, such as confidence, established conventions, and shared history, are likely to be important for understanding outcomes on the individual and aggregate level.
For starters, recognizing knowability’s inherent limits implies that fundamental considerations – and computations based on them – cannot by themselves account for how market participants make decisions. As Keynes put it,
We are merely reminding ourselves that human decisions affecting the future, whether personal or political or economic, cannot depend on strict mathematical expectation, since the basis for making such calculations does not exist; and...that our rational selves [are] choosing between alternatives as best as we are able, calculating where we can, but often falling back for our motive on whim or sentiment or chance.
For Keynes, unlike for behavioral economists, reliance on psychological factors in decision-making is not a symptom of irrationality. Rational individuals in the real world use knowledge of facts; but, because their knowledge of how outcomes will unfold over time is inherently imperfect, calculation alone is insufficient for decision-making.
Although Keynes emphasized that psychological considerations, such as market sentiment, play an important role in individual forecasting, he also pointed out that “we should not conclude from this that everything depends on waves of irrational psychology.” Likewise, Frydman and Goldberg have argued that psychological considerations themselves could not sustain the recurrent long swings that we observe in asset prices. Indeed, empirical evidence concerning the role that fundamental and psychological factors play in participants’ trading decisions in the US stock market suggests that changes in fundamental factors strongly influence how confidence and other sentiments unfold over time.
Beyond psychological factors, Keynes also emphasized that imperfect knowledge leads each market participant to rely on social conventions when ascertaining how other participants might think about the future course of outcomes. As Dow put it in discussing Keynes’s approach to understanding individual decision-making,
Individuality or agency allows for individual choice as to whether or not to follow social convention. But sociality means that social-conventional judgment provides the norm, such that expectations are formed interdependently with expectations in the market. This [non-routine] social interactionism is a key ingredient of Keynes’s…view of the economic system.
The complex interdependence between fundamental, psychological, and social considerations in determining how market participants forecast outcomes and make decisions suggests that these influences can at best be represented with qualitative and contingent conditions. After all, even if there are some regularities in how these considerations influence an individual’s forecasting, and more broadly her decision-making, we would not expect such regularities to follow fully predetermined rules or to persist indefinitely. We would also expect that such regularities are likely to be context-dependent.
The importance of psychological and social factors in individuals’ decision-making also goes a long way toward explaining behavioral economists’ remarkable empirical success in uncovering the gross inconsistencies of conventional models based on a priori considerations. Once economists decided to look for evidence of how individuals actually behave, rather than assuming that they need only identify a set of a priori assumptions that would characterize how supposedly “rational” individuals should behave, the empirical failures of such assumptions in characterizing forecasting or preferences became plain.
4. Rethinking the Scope of Economics
The empirical failure of models that appeal to a priori assumptions about rational behavior, and the shift to reliance on representations based on empirical regularities, undermines the widespread belief that contemporary economics can rigorously explain the findings of other, “soft” social sciences. In fact, the shift away from a priori assumptions requires that economists incorporate findings from psychology and other social sciences in constructing more empirically relevant models.
IKE makes use of behavioral economists’ and other social scientists’ empirical findings to specify the representations of market participants’ decision-making that underpin its accounts of aggregate outcomes. However, unlike the behavioral-finance approach, which represents with mechanical rules its findings concerning how individuals actually behave, IKE formalizes these empirical regularities with qualitative and contingent conditions.
For example, psychological studies have uncovered much evidence that individuals revise their beliefs in the face of new evidence gradually. In modeling exchange-rate swings, Frydman and Goldberg formalize this finding in terms of qualitative and contingent constraints on how market participants revise their forecasting strategies. They show that, despite the model’s openness to non-routine change, it implies that the exchange rate tends to undergo long-lasting swings away from and back toward benchmark values.
Frydman and Goldberg provide another example of IKE’s use of behavioral findings. They show how Kahneman and Tversky’s formulation of preferences, so-called “prospect theory,” can be applied in models that are open to imperfect knowledge on the part of market participants and economists.
However, the importance of the social context implies that, in searching for empirical regularities that might be useful in modeling an individual’s decisions, economists will need to look beyond insights from psychology. For example, Frydman et al make use of Keynes’s insight that social conventions play an important role in individual decision-making in asset markets. They also draw on our understanding of the qualitative regularities that have characterized aggregate outcomes, and suppose that market participants, too, must be aware of these regularities when they form their forecasts. For example, exchange rates’ tendency to undergo long swings away from historical benchmark levels, and then to exhibit sustained counter-movements, plays a key role in their model of the uncertainty premium in currency markets.
5. Probing the Frontier of Formal Macroeconomic and Finance Theory
Although the application of IKE to financial markets appears promising, it is too early to substantiate its broader usefulness in macroeconomic and policy modeling. If qualitative and contingent regularities can be established in contexts other than asset markets, IKE can show how to incorporate them into mathematical models that can be confronted with empirical evidence. However, when change on the individual and aggregate levels, particularly revisions of participants’ forecasting strategies, cannot be adequately characterized with qualitative and contingent conditions, empirically relevant mathematical models may be beyond the reach of economic analysis. In this sense, IKE explores the frontier of what formal macroeconomic and finance theory can deliver. How far, and in which contexts, this boundary can be extended is the crucial open question.
Non-routine Change, Imperfect Knowledge and Economic Science