I want to collect some insights from Data scientists, ML engineers, Data engineers working on AI/ML problems.
What are the main hurdles which slow you down to ship your models to production?
I have summarised some learnings based on my conversations with some folks I talked to.
- No clear agreement on metrics to optimize.
- Necessary data is not getting tracked. Getting this into the app would need another release cycle and user adoption of the newer version which is could be several weeks.
- The superior model in offline evaluation does not guarantee better performance when rolled out in production.
- Not involving engineers during model design and development. Discovering scalability issues late in the cycle.
- Latency constraints impose some restrictions on online (request time) computations
- Starting with a complex model and a large set of features
- Not considering negative side-effects on other KPIs
--
Over to you:
What are the main hurdles which slow you down to ship your models to production?
Image source: mk_is_here (flickr)
No comments:
Post a Comment