A Blog on Analytics and Marketing

SAS, Marketing, Predictive Modeling, Statistics

Ten Reasons Why Models May Fail

Posted by phillippeng on September 14, 2009

Kent Leahy and Nethra Sambamoorthi list ten most common reasons why predictive models in marketing may fail. These top ten reasons are:

(1) Modeling strategy design. The person who will actually be building the model is not included in the initial discussions or design of the model.
(2) Model overfitting. The model has been “overfit” to the sample at hand ,and, consequently, does not generalize well to the actual mailing population, or is otherwise unreliable.
(3) Population shift due to environment changes. The circumstances surrounding the actual mailing change or the mailing environment turns out to be substantially different from the one on which the model was built.
(4) Model generalization too much. The model is used as though it were ‘generic’ or ‘universally applicable’.
(5) Population shift and model overfitting. Changes in the mailing environment in conjunction with the use of an ‘overfitted’ model.
(6) Model out-of-date. The model contains “post-event” variable(s), or those that occurred after the event you are trying to predict.
(7) Model validation and implementation. Not ‘test-scoring’ the model, or making an error when implementing the model.
(8) Sample selection QC. Failing to run an audit of the file as the first step in the model-building process.
(9) Miss the model expectation. A consensus on just exactly what the model is expected to predict (and for which audience) is not reached and/or well understood.
(10) Poor fanancial Planning. The model performs well but the mailing itself is not a financial ‘success’.

Reference: http://www.crmportals.com/crmnews/2002123.html


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: