In my previous life, I had a brief stint as a phd student in the early ‘00s. I utilise my experience gathered from my attempt to gain a title to draw comparisons to the industries I worked in (Finance, Mobile Gaming and Fintech) on methodology and principles.
Machine learning’s academic origins have a significant imprint on how we approach problems and solutions in the field. This is exacerbated by the success of flagship AI shops that have very strong academic roots and operational models. Tools, methods and metrics that work for them might not be suitable for a data driven company.
Furthermore, being a relatively new field, practitioners are still early in their career and their studies and the way universities teach the subject sets their expectations on what machine learning is and how to do it.
This is not a critique of academia, but if you realise you are inadvertently applying someone else’s methodology without their circumstances you need to reconsider your position (see also: You Are Not Google). Lets review key differences between academic and industrial settings:
Goals, Timespan and Feedback
Academia focuses on publishing and the related publicity. This is best achieved by producing work that improves state-of-the-art (SOTA) results on a small set of well understood topics to create comparisons and communicated in short scientific articles. The format restricts the scope of the solutions otherwise it won’t fit into a few pages. Focus is on novelty and not repeatability. Publishing happens relatively infrequently, so feedback is scarce.
Business focuses on improving several frontline KPIs at once in a repeatable manner. The problem the business is facing is sprawling with many stakeholders and external factors. Solutions are repeatedly productionised and provide continuous feedback on changes. This is an ongoing process, businesses never stops improving their own KPIs.
People, Process, Technology
In academic settings, compute is cheap but (external) labour is expensive. Improvements are achieved primarily by adding more complexity and using more computing power. This yields solutions that are fragile to changes in their circumstances when applied to other problems. A research team consists of few and similar background persons so communication can be highly technical.
In a corporate environment solutions must happen in heterogenous cross functional teams. Labour is accepted as given if there is an economic need and communication happens at various technical, domain and business levels. Switching between expert and non-expert communication levels is key.
This requires simple and robust solutions otherwise the productionisation and ongoing maintenance of it will be infeasible. This also helps reasoning and communicating about these systems and enable involvement of all participants regardless of technical level.
Summary
To conclude, academia improves SOTA by additional complexity in few well understood tasks with little human resources, infrequent feedback and in medium term. Businesses focus on many interconnected metrics/KPIs on convoluted, end-to-end products with large cross functional teams, under frequent feedback and very long term operations.
Instead of improving arbitrary metrics with fragile solutions, industrial machine learning must focus on mapping the cross functional domain knowledge to robust and maintainable solutions on an incremental basis.
Please subscribe to see how.