All models are wrong, but some are useful” – applies to project planning as much as it does statistics. Especially when determining a plan for doing innovative new work, determining how long it will take can easily become divorced from reality.
I find that the “Rule of Three” is usually a good model to start planning projects. For any given piece of work, it is helpful to estimate three equal-sized portions for planning, building, and testing.
When projects fail, it tends to happen due to the under-estimation of the plan or test phase. Just as the circumference of a circle divided by its diameter will always be pi, the time it will take to do a thing will usually be half the time it takes to plan and test that thing. Skip planning or testing to speed the thing up, and it will probably fail. It will fail by being the wrong thing. Or it will wail by being a thing that doesn’t work as it should. Ultimately the time spent fixing the thing will end up longer than build times three.
Many feel that building is the fun part where we do what we do best – be it coding, constructing, cooking, performing, or writing. Planning and testing inject purpose into the work and confirm the outcome meets the purpose. They require creativity and skepticism. They also slow us down. It is tempting to cut corners on these activities which aren’t as visible as the actual doing of the thing, but short cuts come at a cost.
How can we be agile and move fast if everything we build takes 3x as long? Fortunately, while the rule dictates proportion, it does not determine order. The agile approach to the rule is to shrink everything down – faster planning, smaller build, and quicker QA. These iterations satisfy the rule of three while also shortening the time to delivery. Additionally, if small chunks of work are common, the entire plan, build, test cycle reduces once the team has experience knowing what to do. Finally, small chunks of effort can be done in parallel, further shrinking the time (although increasing the resource budget).
There are many ways to apply the rule and variations that apply beyond the fields of software engineering or data science. When you catch something that is in exception to the rule it is worth considering – have planning or testing been under-done? Maybe it will save you a lot of time and difficulty when the estimate models all turn out wrong.