Causes of Machine Learning’s productivity problem
Summary
“Productionised machine learning is akin to building a plane out of parts while falling out of the sky. While also training the mechanics on how to use a torque wrench. While the torque wrench is on fire. And you and your boss are on fire.“
Over the last week, I wrote a five-piece series on potential causes of Machine Learning’s productivity problem. This article is a summary and a reference to the collection.
A successful endeavour requires three main components: strategy, resources (people and technology) and a way to evaluate if the second executes the first as intended (metrics and processes).
Let’s go through each category in order and examine what issues can affect productivity in Machine Learning projects.
Strategy
As a new and hyped field, Machine Learning can be a confusing space. Many attempts to solve fail because they are not backed by a viable strategy:
Causes of Machine Learning’s productivity problem: Strategy
Teams
Everyone is a beginner in a new field and you need to take this into account. Hire people to complement the rest and be willing to learn and improve.
To quote von Clausewitz:
"You win wars with the army you have, not the army you want."
Causes of Machine Learning's productivity problem: Teams
Technology
"Build vs Buy" vs "Build THEN Buy". Prepare for a full-stack experience. Avoid quick fixes and miracle products:
Causes of Machine Learning's productivity problem: Technology
Metrics
“If you can’t measure it, you can’t manage it.” - Peter Drucker
It’s amazing how many projects start without a clear assessment of their impact (both positive and negative) on the business. You need to connect these to statistical metrics and only then you can expect success.
Causes of Machine Learning’s productivity problem: Metrics
Processes
ML is a new area, and most practitioners freshly arrive from other fields. ML, as of today, doesn’t have a standard way of “doing things”. If you don’t have standards, people will make up their own based on previous experience, which can lead to applying principles that are valid in their own field but not in Machine Learning.
Causes of Machine Learning’s productivity problem: Processes
About the author
Hi! Thank You for reading my blog! I am Laszlo, ex-quant, ex-Candy Crush data scientist, recovering startup founder. I run Hypergolic a Machine Learning consultancy in London. We help startups and enterprises to get ML right on their first try. Our mission is to make Machine Learning less stressful and frustrating for all participants (tech or non-tech). I regularly write on the topic of best practices, best technologies and the latest news in ML.
I am also passionate about teaching Data Scientists to write better code. Please, continue to the series on the topic: https://laszlo.substack.com/p/refactoring-for-data-scientists-how
I hope you find this reference post useful, and if you do please feel free to share it. If you would like to be notified of my writing in the future please subscribe: