When I first saw the NYT 4th-Down Bot last year my reaction was, now maybe we’ll make progress. Successful analytics is about far more than whether or not your numbers are right. It’s an exercise in persuasion. I figured the bot could only help. New angle, new platform. But if the past is any guide, it will be an uphill battle.
Economist David Romer first published in 2002 an early version of his now famous paper on optimal 4th-down decisions. In 2004 David Leonhardt wrote a NYT piece on Romer’s research. And in 2006 the august Journal of Political Economy published the final version (under the title “Do Firms Maximize?” – Romer is clearly trying to influence a different audience). Every football analyst has known this paper for years. And I’m sure someone with virtually every NFL teams knows it. Even Belichick claimed to have read it. But has it made any difference?
We had cause to take a look at the evidence over the weekend. This is just a quick-and-dirty take, and there are many other details that could be sussed out. But at a high level we see very little change over the last 15 years. On 4th-and-short (3 yards and less) there has been some interesting movement. In fact, as of 2009 you might have thought Romer had changed the world in this narrow little way, with the probability of going for it rather than punting rising above 30% (excluding 4th quarter, to focus on the general philosophy rather than response to game-deciding series). But over the last 5 years coaches have retreated to pre-Romer levels in the mid- and lower-20s. 3rd-and-longs are even more grim, with a slight but almost steady decline since 2000.
This seems to confirm Coach Marrone’s claim last week that his conservative, bot-defying 4th-down decision-making is in-line with his colleagues. There is much work to be done.
