Hurdles in shipping data science models to production

I want to collect some insights from Data scientists, ML engineers, Data engineers working on AI/ML problems.

What are the main hurdles which slow you down to ship your models to production?

I have summarised some learnings based on my conversations with some folks I talked to.

• No clear agreement on metrics to optimize. 
• Necessary data is not getting tracked. Getting this into the app would need another release cycle and user adoption of the newer version which is could be several weeks.
• The superior model in offline evaluation does not guarantee better performance when rolled out in production.
• Not involving engineers during model design and development. Discovering scalability issues late in the cycle.
• Latency constraints impose some restrictions on online (request time) computations
• Starting with a complex model and a large set of features
• Not considering negative side-effects on other KPIs 

--

Over to you:

What are the main hurdles which slow you down to ship your models to production? 

Image source: mk_is_here (flickr)

Book summary: Show your work

 I just finished the book `Show your work!` by `Austin kleon`. #showyourwork

Book link from the author: https://austinkleon.com/show-your-work/

Taking inspiration from the idea proposed in the book; I am reinstating my old blog where I was attempting to do something similar in the past. Let us see how far does it go. 

My main take-aways from the book:

1. Creativity is not limited to the few great genius like Mozart, Einstein, Picasso. It can be extended to anyone with "scenius". Under this model, great ideas are often result of collective effort by group of creative individuals who make an "ecology of talent".
2. Be amateur. You would have nothing to loose. You won't be afraid to make mistakes in public. Amateurs know that contributing something is better than nothing. 
3. Start documenting your work. You would have more clarity about what you are working on. Also, you would have lot material to share when you have the results.
4. Share your learnings. You would gain more by sharing.
5. Give proper credits/links/citations for your sources. 
6. Online sharing might have a risk of trolling. Learn how to not to feed trolls. 
7. It is OK to ask for money if you are creating value through your project. 
8. Pay it forward your success by creating opportunities for the folks who supported you. 
9. Work like a chain. Collect feedback/thoughts for your current work before you jump to the next project.
10. Begin again. When you feel like you have learned enough about what you are doing change a course and find something new to learn about. 

P. S: 

Thanks to 

1. My mentor Ankur Kaul for the book recommendation.

 2. VOEBB (Verbund der Öffentlichen Bibliotheken Berlins) for making it available to me instantly on the same day. (For #berlin folks I would highly recommend if you into e-books reading).