Economists who don't do it with models

"Economists do it with models. But those models are generally either really old or really ugly." - Noah Smith, attempting to be witty

Scott Sumner says: Formal models? Economists don't need no steenkin' formal models!

There are basically two types of demand-side models, both of which are nearly useless: 

1.  Some general equilibrium models are used to find which stabilization policy regime is optimal from a welfare perspective.  Most of these models assume some sort of wage/price stickiness...The problem is that...You can get pretty much whatever policy implication you want with the right set of assumptions.  Unfortunately, macroeconomists aren’t able to prove which model is best... 

2.  A second type of model tries to show how to best implement a specific type of policy regime, like inflation targeting...Unfortunately, these “implementation models” conflict with the EMH—it’s not clear why the central bank wouldn’t just peg the price of a futures contract linked to the goal variable... 

To summarize, despite all the advances in modern macro, there is no model that anyone can point to that “proves” any particular policy target is superior to NGDPLT.  There might be a superior target (indeed I suspect a nominal wage target would be superior.)  But it can’t be shown with a model.  All we can do is construct a model that has that superiority built in by design... 

Models are toys to show our students.  When we face serious real world dilemmas it’s time to put away the toys and get real[.]

Sumner essentially restates my big critique of modern macro: How do we know which model to use at any given time? There are a ton of macro models but no commonly accepted scientific standard for validating/rejecting them. So what we (the profession, the Econ Hive Mind, whatever) end up doing is choosing a model based on how plausible we think its assumptions are. We go with our intuition, gut feeling, or politics instead of data. Models are "stories", and we choose the story that sort of sounds the best.

But what should we do instead? Here is where Scott and I part ways. Scott basically says: Why constrain your story with math when you can just tell the story? And I say: Sure, but you're still just telling stories.

There are advantages to telling your stories in English instead of in Math. One is that you can be more flexible - you can add stuff to your story, or be ecumenical in your view of the world, without being worried about pesky internal contradictions. The other advantage is that you can explain your ideas to a wider class of people, who either don't understand Math or can't be bothered to read through it.

Of course, there are disadvantages as well. One is that you can't give quantitative predictions: If we deviate from NGDP Level Targeting by X%, how much will our incomes go down? And so forth. This problem can be solved by using simple "ad-hoc" or aggregate-only models, which are fairly easy for most people to grasp and can be modified without much effort. Then there's the second disadvantage of English storytelling, which is internal consistency. Formal models rely on their assumptions, but once you choose your assumptions, your conclusion is forced by the Math. With English storytelling, you can state conclusions that don't follow from your own assumptions, and often no one will be the wiser.

But I feel like this list of advantages and disadvantages ignores the elephant in the room: How do we know if the story is right? (For you George Box-quoting jerks, what I mean is: How do we know how usefully this story can be applied in a given situation?) Whether you tell your story in complex Math, simple Math, or English, it is still desirable to have a way to evaluate the story.

How do I know that NGDP Level Targeting (Scott Sumner's cause celebre) is a good policy? It hasn't been tried before. So to evaluate how good of a policy it would be, we need to know a lot about how the economy works. Scott Sumner has some ideas (informal models) about how the economy works. How well do these ideas match reality? I'm not sure. And more importantly, I don't know how I'm supposed to be sure. If Sumner used formal models, I could at least go and do a statistical test of the model (for example, a Full Information Maximum Likelihood test) against the available macro data. Sure, no one would pay attention if the test rejected the model - the data are crappy and the econ profession lacks generally agreed standards for scientific disproof - but at least I could do it. But since Scott's stories about the economy's inner workings are not quantitative and thus not subject to data, I don't even know how to evaluate them.

I mean, sure, I can think of some extreme stylized facts that, if true, would put the lie to Scott's basic story - for example, if inflation tended always to rise in recessions - but to require such a huge, glaring discrepancy in order to invalidate a model is setting the bar for your theory pretty low (in the language of econometrics, it's "protecting the null hypothesis").

In other words, the basic, fundamental problem with macro, as things stand, is that it's not scientific. Making it less formalized, or less mathematical, doesn't get around that problem. And it seems pretty likely to me that if macro ever does come up with a way to tell if models are right, it's going to require that those models be of the formal mathematical type.

Update: Though, come to think of it, biology models are not mostly math. If you give a description of how the liver works, the description won't rely on equations, and in fact will be more useful than one that does rely on equations. So maybe macroeconomic science won't rely on math; maybe it'll rely on qualitative descriptions of the function of key components of the economy. I'm really not sure. But in any case, we need some standard of model evaluation...

Update 2: Scott Sumner gets model-icious!