Football Forecasting - Introduction - EdsCave

I am not a football fan, or even a sports fan.  How I got interested in football forecasting was that a few years ago I was running a small consulting business, with a focus on providing quantitative analytics and modeling methods to help improve marketing effectiveness.  I was out to dinner one evening at a local bar and I was thinking about how to drive traffic to my site.  Athough the bar was not primarily a 'sports bar', there were several screens tuned in the NFL game in progress, and it occured to me that putting up football scoring predictions might be something that would attract some attention. After all, a great many potential clients would also be sports fans, and providing a weekly forecast would certainly be a good demonstration of having at least some competence in the are of predictive modeling - even more so since I know next to nothing about the NFL teams, don't follow the sport or even watch the games, and have next to no knowledge of the game itself. This would be almost the ideal advertisement for being able to take on predictive analytic or business optimization problems where one of the big client objections was that 'you don't know anything about our business'.  If I could demonstrate calling games accurately without knowing anything about football, I could certainly help you optimize your marketing efforts!

Well, this theory didn't play out very well, mainly because over the course of a year of knocking on doors I came to realize the following:

• Analytics and optimization are hard sells at a small-to-medium sized business. At a railroad company buying hundreds of millions of dollar of fuel a year, a 1% improvement results in serious savings. It is much harder to make the case at a small business that may not have a total revenue of a million dollars.  
  

• The whole notion of analytics is not all that trusted.  To put it bluntly, it changes the nature of how expertise is viewed and utilized.  For a simple example, consider that when Google decides to make a button a new color, it runs experiments with its user base, and bases its decision on the results of those experiments. In contrast, other businesses may rely on the intuitions of their website designer.  Bringing in analytics to an organization can change the roles of many professionals, sometimes in ways that they may not be too thrilled with.
  

• The kinds of data that are likely to yield the most value when analyzed are those that are also likely to be considered the most proprietary - material which would often be considered too sensitive to release to an outsider even under the most draconian non-disclosure agreement.
  

After a mostly unsuccessful year of trying to peddle my analytics expertise, I closed the shop and bagged the forecasts. A few years later at one of the INFORMS conferences I ran into a group of sports analytics enthusiasts (SpORts special interest group), one who I recall having a conversation about game forecasting and discovered we were both using very similar methods.  

Another thing that kept me interested in the 'football problem' was the paper 'Fast and Frugal Forecasting', by  Goldstein and Gigerenzer.  At the time I was competing in analytics competitions and noticed that to win, you typically needed ensembles of ungodly complex algorithms and analysis. To come close, however, some pretty simple models often did the trick. In Goldstein & Gigerenzer's paper, they observed the same kinds of effects, except for forecasting problems with high uncertainty, simple methods often outperformed far more complex ones. Professional sports games certainly qualify as a 'high uncertainty' situations, which also happen to have lots of recorded stats. The availability of this data makes pro sports a great test platform for analytic techniques.

I have experimented with a few very simple algorithms for predicting the next week's NFL game outcomes based only on the score information from the season up to the present week.  While there is a plethora of statistics about games and players, I chose this limited input data set as it was readily available and lent itself for use by simple (in some cases mind-numbingly simple) forecasting algorithms.  One of the points I wanted to make was that simple can also be pretty effective - and so in the summary page I compare the results side-by-side.

I had a bit of fun thinking about this problem and putting these pages together, and I hope you also have a bit of fun reading them.

Next Page - Home Team Wins