Home Forums Kamanja Forums Data Science & Models Production Model Lifecycle Management – presentation

This topic contains 0 replies, has 1 voice, and was last updated by  Greg Makowski 1 year, 4 months ago.

  • Author
    Posts
  • #18741 Reply

    Greg Makowski
    Moderator

    Hello,

    This was a presentation I gave last week.  I was posting it here to invite back-and-forth discussion.  People may have valid, different approaches.

    Production Model Lifecycle Management

     

    q1) I am aware of other ways to extract out “variable importance ranking” – please share links or describe other ways you use.  What are pro’s and con’s?  Some approaches are algorithm specific, some are algorithm agnostic.

    q2) How do you provide “record or person level reason codes” when needed by the application?  I am aware of score card modeling for linear methods.  How would you deal with non-linear models, with ensembles of models?

     

    q3) How do you manage the lifecycle of models that you have had in production?  Do they get replaced after a fixed time (i.e. a day for banner ads, a month, a year, …?).  Or if you use data driven metrics, what metrics do you use?  Do you use a strict policy or threshold, or allow some leeway, based on available resources on when to start refreshing a model?

    q4) When replacing a model, do you always fully replace it?  Do you ever just do a minimal retrain on fresher data, for a “model refresh”?   Do you have any other choices for how you update models?

     

Reply To: Production Model Lifecycle Management – presentation
Your information: