Documentation vs Documentation in Data Science

Posted on Linkedin on 2022-04-20

People usually talk about the importance of documentation and immediately jump to a conclusion about the tool to use and the level of detail to document (extremely detailed) regardless of context.

This immediately makes any documentation attempt a "big deal" and a chore. You put it away as long as possible because no one wants to do it. Then, of course, it won’t be done. Or do it grudgingly, leave it as it is and go stale in two months.

Documentation should be a lightweight, high ROI activity, and a productivity tool.

There are three types of documentation you need to do:

• Documenting work (business cases, results, ideas)

• Documenting code (your own implementations)

• Documenting modules (share it on PyPI, Github, ReadTheDocs etc.)

Documenting work

Your boss will not read API documentation generated by Sphinx. They require "presentation style" reports, “result driven” is the key phrase. To help your colleagues connect with your work, you need to succinctly summarise what’s the context, what did you try, what worked and what was the solution in the end.

Use some internal wiki or slides for this. Watch content from management consultants on how to do that best. This is their edge.

Documenting code

Your code should document itself. Learn how to maintain readability.

There is no need for comments like this:

𝚍𝚎𝚏 𝚊𝚍𝚍𝟷(𝚡):
  """𝙰𝚍𝚍𝚜 𝚘𝚗𝚎"""
  𝚛𝚎𝚝𝚞𝚛𝚗 𝚡+𝟷

You need to write readable code rather than readable documentation. If you still feel the need to document focus on the “Why” (the only one of “Why-How-What” that is not in the code) but at this point take a good look at the first point if it is not better placed there as a blogpost.

Documenting modules

What you need to document with Sphinx-like tools are "published interfaces". Modules with code that are too difficult to read each time. Or ones that are used by people who do not have access to the codebase.

Documenting is an 80/20 activity

Documenting your work should be a value-added activity, not just a chore. If you hit the right level, you and your team will fill empowered by doing it. Iterate on it until you reach this state.