Leadoff Rant: over-cited prediction models are mostly useless

Leave a comment

March 25, 2013 by Nick L.

*Note: all screen shots are from Baseball Prospectus’ 2012 MLB Preview.

With just one week remaining until opening day I think it’s finally time to get into a little season preview action. To be honest, I’m not the biggest fan of previews, or any of the numerous prediction models that websites- like Baseball Prospectus- wheel out to tell us where our favorite teams should finish the year. Boston

It’s next to impossible to model for everything that will happen to any given team throughout the season, and even when these prediction models succeed in accurately predicting a team’s final record, its due more to luck than any sort of statistical genius. In baseball, good teams will usually have between 88 and 95 wins, and very, very good teams might get to 100 wins. A casual baseball fan could probably forecast the final records of the game’s strongest teams just as accurately as a prediction model, without all the complex mathematics.

So when a prediction model tells you that a team will finish the year with 95 wins, and the team ends up winning 96 games, all that tells me is that the team was a very good team coming in to the season (usually something you can ascertain before the season with a casual glance at the previous year’s stats), and they played like one throughout the year. Baltimore

Where you see major Bear Stearns-type failings in prediction models is when teams like the 2012 A’s and Orioles drastically outperform the models, or teams like the 2012 Marlins drastically underperform what the models said they would do. Before the 2012 season all of these models had the Marlins at or near the top of the NL East, and the Orioles and A’s at or near the bottom of the AL East and West. If Baseball Prospectus were an investment bank they would have been groveling for a bailout long ago. Washington

Even when these prediction models succeed in coming close to predicting a team’s actual record, it’s not because what the team did during the long regular season grooved perfectly into the contours of the model. That would be almost impossible. Baseball is too unpredictable, and the season is too long. As I pointed out before, good teams usually end up with between 85 and 95 wins, and good teams will usually get those wins- but not in the exact way prediction models say they will. Oakland

That’s not to say that prediction models aren’t fun to look at and argue about. They are. But let’s stop treating them like they’re gospel.

The economics profession realized long ago that they’re better at explaining why things happen than they are at predicting what will happen in the future. The sabermetric community appears to be too busy trumpeting their “revolution” to take the time to indulge in any sort of similar self-reflection. Miami

That being said, I’m about to do my own MLB preview because, hey, the season is still a week away, we don’t have any meaningful games to discuss, and we need to do something to satisfy our collective baseball jones’ (or at least I do). My preview will be utterly devoid of any scientific rigor or concrete predictions. I promise.

Baseball is unpredictable- not pretend like it isn’t.


Okay, I’m done ranting. First up: the NL West.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: