I can't summarize Kocherlakota (just read it) but the key critique is as follows
To an outsider or newcomer, macroeconomics would seem like a field that is haunted by its lack of data, especially good clean experimental data. In the absence of that data, it would seem like we would be hard put to distinguish among a host of theories with distinct policy recommendations. So, to the novice, it would seem like macroeconomists should be plagued by underidentification or partial identification.I think the prevalence of puzzles is no more puzzling than the fact that the squares in a crossword puzzle aren't filled in. The puzzles are the point of the exercize. In particular, researchers need problems which are neither to easy nor to hard. Easy problems have been solved decades, centuries or millennia ago. Hard problems won't be solved before tenure decisions are made. The macroeconomic puzzles are just difficult enough. Furthermore the huge gap between the standard DSGE model and reality means that the solution to one puzzle creates another puzzle.
But, in fact, expert macroeconomists know that the field is actually plagued by failures to fit the data – that is, by overidentification.
Why is the novice so wrong?
The answer is the role of a priori restrictions in macroeconomic theory.
The mistake that the novice made is to think that the macroeconomist would rely on data alone to build up his/her theory or model. The expert knows how to build up theory from a priori restrictions that are accepted by a large number of scholars. (Indeed, in the academe, that’s exactly what it means to be an expert macroeconomist.) Those restrictions are what give the models their empirical content. As it turns out, the resulting models actually end up with too much content – hence, the seemingly never-ending parade of puzzles.
JEC at Mean Squared Errors puts it much better than I could
Consider the macroeconomist. She constructs a rigorously micro-founded model, grounded purely in representative agents solving intertemporal dynamic optimization problems in a context of strict rational expectations. Then, in a dazzling display of mathematical sophistication, theoretical acuity, and showmanship (some things never change), she derives results and policy implications that are exactly what the IS-LM model has been telling us all along. Crowd -- such as it is -- goes wild.
And let's be clear: not even the most enthusiastic players of the macroeconomics game imagine that representative agents or rational expectations are, in any sense, empirical realities. They are conventions, "rules of the game." That is, they are arbitrary difficulties we impose on ourselves in order to demonstrate our superior cleverness in being able to escape them.
I add, however, that some assumptions are made, because it is too difficult to manage without them. We know that we don't have rational expectations, but we don't know how we actually do form expectations. The macroeconomist can't wait for psychologists (and sociologists) to finish their research program. Efforts to relax the rational expectations assumption have ended either with even less plausible mechanical assumptions or with extreme mathematical difficulty in models which are otherwise as simple as possible.
As usual, the challenge to the young macroeconomist goes back to Keynes. The General Theory of Employment Interest and Money begins with book 1 containing models which are not difficult enough (aside from the fact that they were explained clearly and in detail a year later by Hicks). It also includes Chapter 12 on long term expectations (beauty contests and all that) clearly presenting problems too hard for Keynes. Then thereare warnings really to not trust a Phillips curve (not called that 23 years before Phillips) and discussion of how wage setting is an extremely complicated and probably incomprehensible social process. So there are the parts which were finished at least 79 years ago and the parts which might maybe be finished 79 years from now (but I doubt it)
Here (as almost always) I am following Krugman
I’d divide Keynes readers into two types: Chapter 12ers and Book 1ers. Chapter 12 is, of course, the wonderful, brilliant chapter on long-term expectations, with its acute observations on investor psychology, its analogies to beauty contests, and more. Its essential message is that investment decisions must be made in the face of radical uncertainty to which there is no rational answer, and that the conventions men use to pretend that they know what they are doing are subject to occasional drastic revisions, giving rise to economic instability. What Chapter 12ers insist is that this is the real message of Keynes, that all those who have invoked the great man’s name on behalf of quasi-equilibrium models that push this insight into the background – from John Hicks to Paul Samuelson to Mike Woodford – have violated his true legacy.
The lack of puzzles is due to the fact that the puzzle addressed in book 1 is solved in book 1 and the problems posed in bok declare themselves to have "no rational answer". Neither provides a good dissertation topic. They do provide useful policy guidance. Keynes wrote that there will be manias, panics, and crashes no matter what policy makers do, and that they can deal with the aftermath by temporarily increasing government investment or consumption. If that's as good as macroeconomics can do then it's a very boring research program which reached it's apex the day it started. But it sure would be better than what ma macroeconomists did in 2008 with internal debates which weakened already limited influence on policy makers who did crazy things.
I am now going to write a long boring unoriginal post on which problems in macroeconomics are too easy and which are too hard. Aggregate demand is too simple. An IS curve works very well. Economists can create a problem by arguing that it shouldn't work and declaring its success to be a puzzle and others can let us have our fun. But they would be foolish to pay us for our play.
The accounting identity is that aggregate demand is equal to private consumption + private investment + government consumption + government investment + exports - imports. In standard DSGE models the last four terms are not microfounded. there is just a random variable for GDP minus private consumption minus private investment. I don't think it is wise to ignore foreign trade (especially not when trying to apply models to countries other than the USA). Here I think the problem is that exchange rates fluctuate wildly. Those fluctuations matter, but no one has found a rational explanation. So either one accepts irrationality or one leaves the effects as an undiscussed unexplained disturbance term. In particular and importantly, aggregate consumption is much much too simple. It can be fit using current disposable personal income and any crude measure of wealth (ordinary non-human wealth). It helps to add a few lags of current disposable personal income. This is rather important, because it has a huge effect on estimated multipliers and the lags are needed to fit the response to shifts in fiscal policy (that is get a multiplier of around 1.5 and not around 8). My view is that Keynes said ust about everything useful to be said about aggregate consumption with two exceptions. He didn't discuss circulating credit and living off of credit cards (because they didn't exist then) and he didn't recognize the important role of housing equity as one of the key forms of wealth which affects consumption.
I do seriously think that list of omissions is more or less adequate and complete. I argue this at great length here.
Now this is interesting, because Friedman's permanent income hypothesis has a central role in contemporary DSGE macroecomics. One of the two dynamic equations corresponding to the D in DSGE is the Euler equation, which has no place at all in Keynes. It also just so happens that the implications of the Euler equation are a leading source of puzzles and it is, by now, widely hated even by the open minded Simon Wren-Lewis who wants to give DSGE the benefit of every doubt and the very central (states) New Keynsian who works near a great lake Martin Eichenbaum. Keynes's argument was that future aggregate income is very hard to predict, so while the ratio expected future income to current must be considered when studying individual consumption, it is can and should be neglected by macroeconomists. I know of no evidence against this position.
Investment is much trickier. It is the topic of the dread chapter 12 (among others). It is, in fact, hard to fit and harder to forecast private investment. Here I think a large part of the problem for DSGE is the decision to model investment as a scaler and identify all of it with business fixed capital investment. Residential investment and inventory investment are not included in standard DSGE models. This is odd, because inventory investment is discussed constantly by practical forecasters who talk about the latest business cycle. Also it should be clear since 2008 that housing investment is extremely important.
I report some 1960s style econometrics of investment here. Here I learn (from Krugman) as usual that useful knowledge was abandoned in the current dark ages of macroeconomics
Back in the old days, when dinosaurs roamed the earth and students still learned Keynesian economics, we used to hear a lot about the monetary “transmission mechanism” — how the Fed actually got traction on the real economy. Both the phrase and the subject have gone out of fashion — but it’s still an important issue, and arguably now more than ever.Here it is worth noting that while macroeconomists have worked very hard to include financial frictions in DSGE models, they still don't include a housing sector (as far as I know). This is very odd (have I missed a flourishing literature). I have some thoughts as to why housing might be ignored. First back to consumption. Housing wealth does not cause high consumption in a model with a rational representative consumer. The consumer's wealth in terms of consumption goods increases, but so does the price of the consumer's consumption bundle including housing services. Housing equity can relax liquidity constraints with HELOCs but the representative agent can't face a liquidity constraint. Housing equity can reduce precautionary spending as someone who, for example, is laid off can sell her house and live in a smaller house. But the whole economy can't sell its houses.
Now, what you learned back then was that the transmission mechanism worked largely through housing. Why? Because long-lived investments are very sensitive to interest rates, short-lived investments not so much. If a company is thinking about equipping its employees with smartphones that will be antiques in three years, the interest rate isn’t going to have much bearing on its decision; and a lot of business investment is like that, if not quite that extreme. But houses last a long time and don’t become obsolete (the same is true to some extent for business structures, but in a more limited form). So Fed policy, by moving interest rates, normally exerts its effect mainly through housing.
Notably the apparent effect of wealth on consumption (separately from its role as a predictor of future income) was rejected because it is hard to micro found
I would argue we built the first UK large scale structural econometric model which was New Keynesian but which also incorporated innovative features like an influence of (exogenous) financial conditions on intertemporal consumption decisions. I havent read the linked paper. I can see a problem with using stock market indices as wealth which directly affects consumption. They should summarize future dividends which are future income and so already considered in the permanent income hypothesis. Here the problem is that stock fluctuations are very large compared to the tiny fluctuations in the achieved present value of future dividends. If there is noise in stock prices, one might ask if it helps explain consumption and how it deviates from the consumption of hypothetical rational representative agent. There is and it does, but facing that fact requires one to abandon the rational expectations assumption and join one side of a furious debate in finance. Now I think it is important to understand why contemporary models of monetary policy and its effects totally suppress the fact about what sort of investment is affected which Krugman was taught back in the age of the dinosaurs. I suppose it is probably really because it was considered a harmless simplification and it is very important to have few variables if the benchmark of one's benchmark model is a VAR. But I can see a number of reasons why housing is problematic. The first is that there was a huge housing bubble which is very hard to reconcile with the rational expectations assumption. It is much easier to explain a huge increase in stock prices as the effect of an increase in expected future technological progress. Second it is hard to understand how monetary policy affects housing investment. First it is necessary for the monetary authority to affect real interest rates (and that already loses the hard core new Classicals). Then it is necessary that fluctuations in short term interest rates cause fluctuations in mortgage interest rates. I think the high correlation doesn't make sense (so long term bonds and mortgages are good buys when short term interest rates are high). The 30 year fixed interest rate mortgage is a very extreme case of nominal rigidity. Their existence makes no sense. I think that, like foreign exchange markets, housing demand is incomprehensible and must be treated as an unexplained and unpredictable source of shocks. On the other hand, there are useful simple things to say about investment. First it is well fit by a flexible accelerator which gives high investment to GDP when GDO growth is high and low investment when short term nominal interest rates minus lagged inflation is high. In particular, the vulgar estimate of a real interest rate helps explain housing investment (and not so much if at all business investment). The problems are too hard (chapter 12 and business fixed capital investment, housing bubbles and housing investment) or solved long ago. Now the old IS model has stong implications which are very different from those of standard DSGE models. In the old model, there was no Ricardian equivalence. I think there is no evidence of Ricardian effects at all. I think the IS curve works find and the only problems are that it is very old and that no optimization under constraint is considered when writing it. Aggregate supply (coming fairly soon -- lags are important). Part II here
Our efforts impressed the academics on the ESRC board that allocated funds, and we won another 4 years funding, and both projects were subsequently rated outstanding by academic assessors. But the writing was on the wall for this kind of modelling in the UK, because it did not fit the ‘it has to be DSGE’ edict from the US. A third round of funding, which wanted to add more influences from the financial sector into the model using ideas based on work by Stiglitz and Greenwald, was rejected because our approach was ‘old fashioned’ i.e not DSGE. (The irony given events some 20 years later is immense, and helped inform this paper.)
 Consumption was of the Blanchard Yaari type, which allowed feedback from wealth to consumption. It was not all microfounded and therefore internally consistent, but it did attempt to track individual data series.