[Interesting content] 3x Explore, Expand, Extract • Kent Beck • YOW! 2018
[Interesting content] 3x Explore, Expand, Extract • Kent Beck • YOW! 2018
2023-05-02
I always felt guilty that we don’t write unit tests. This was until I found this video from Kent Beck:
I know this will be controversial, so let’s caveat this:
We usually work in experimentation, R&D, and POC projects where everything is up in the air, and you are under huge time pressure.
We still need to manage code quality; otherwise, we will go mad.
We write unittests on the stable parts of the codebase that usually survive the project no matter the concrete project’s outcome: These are typically adapters (as in the design pattern), domain models, and utilities. But this is more software engineering and less data science/machine learning anyway.
What caught my eye in the video is this [7:25]:
“... And I took the scariest decision of my whole career. I realised that the only way to be successful in my career was to forget everything I learnt about software engineering and relearn it from scratch. ... Does it need tests? Of course, it needs tests. ... Apparently, it doesn’t need tests...”
There are several good points:
Nasseem Taleb convex and concave payouts
Convex: Lots of small bets and, once in a while, a huge payoff.
You want as many bets as possible because you don’t know which one will pay off, and you risk, in general, very little.
Concave: Lots of large bets with small margins and occasionally a huge loss.
You want the margin to go up just a little bit but, at the same time, absolutely avoid increasing the chance of a huge loss.
Return of the Waterfall (of course, they don’t call it Waterfall):
Let’s do a thorough market survey before we start because we want to understand the landscape first.
And then, we will do a “business model refinement.”
And then, we will draw what exactly the product will be.
And then some implementation stuff, but we have programmers for that…
Should I say he is not a fan… (Waterfall doesn’t work for well-understood reasons, partly because of a lack of feedback. See also “Problems with ML Lifecycle”)
But he realises that people are trying to avoid the Concave situation from above. Not maximising wins but minimising losses.
Sidenote:
Waterfall doesn’t work because it’s a third type of payoff: Huge investments most of the time (all the preparation before coding), then rarely a large payoff. The rest is a waste. Again, think about ML Lifecycle… It _is_ Waterfall.
Convex and concave curves can be fit together to form a third “S”-shaped curve.
First question: “What do you have to lose?” At the exploration stage, very little, so you need to make lots of bets. And once you make a hit, you find a growth loop where you have a reinforcing effect where if you grow the easier to get to grow more.
Once you have this, you are in the Expansion phase. Each next blocker will kill the growth loop, and you won’t have a sustainable business. So you need a different mindset:
Exploration is about cheap experimentation and coverage of the problem space.
Expansion is about finding the next thing that can kill you and fix it before it does.
If you don’t see the transition from Exploration to Expansion, you will have a hard time. You will probably miss an opportunity. Expansion is short and intense as it is a transition to the next phase:
The third one is Extraction. This is where you have good models and playbooks. You can measure things and make informed decisions. KPIs make sense because you have a lot of data. In Exploration, these don’t make sense because there is no data, and we are not even sure how to measure what is happening with what KPI.
In Extraction, you also start to have negative behaviours. People will game the system. One experiment collides with another. So your goal is good decision-making.
The mistake of XP
And here is the important part [51:30]: The mistake of XP (Extreme Programming, what is now understood as TDD, Agile or just “programming”) was that it tried to be a set of tools for all three parts of the cycle.
“Should you write tests? It depends. Do you have anything to lose? No? Only write the tests that help you experiment quicker.”
If you have a cheap and alternative way to check that your code is still doing what you think it should do, use that during the Exploration phase.
Watch it yourself; it’s great stuff:
And, of course, subscribe: