Why not subscribe?

Thursday, April 05, 2012

The perils of “optimizing” in models

The cartoon is hard to read below, but can be found at http://www.smbc-comics.com/index.php?db=comics&id=2569#comic – which is where Zach Weiner would rather have you read it anyway.

Co-workers are familiar with the fact that I hate to hear somebody say they want to optimize their pricing, or optimize their promotion, or optimize their product assortment.

I think good models can improve things, sure. In part, models can point out where you are doing really dumb things, so you can do average things instead. Or, find average things and do things that are 20% better. That’s not so sexy, but is a something that is much easier to achieve.

“Optimize” is a degree of unfounded optimism or misguided hope that can lead people to explore the far ends of a statistical model, where it is most questionable and most likely to lead to misleading advice.  Or, it might lead to solutions which are optimal for a particular part of the process, but actually less functional for the process in its entirety.

In the cartoon below, the computer model has been programmed to optimize total happiness. This does not turn out well.