Michael Wolfe on Quora posted an amazing analogy explaining why software development estimates are routinely way off, which results in developers padding their estimates to factor in the fudge of such inaccuracies.

Another thesis I’ve been pondering over lately are the terms “Software Engineering” and “Computer Science”, which imply that software development is purely scientific and engineering in nature. Most colleges run those programs out of their science or engineering departments, and the business surrounding technology (budgets, management, and processes) is premised on the assumption that coding is engineering in that it’s scientific, mathematical, calculate-able, and algorithmic.

All of that is a component for sure – but the process itself, the act of coding is much closer to art. You start off with an idea of what you need to do and how you’re going to do it, you start roughly putting it together, you realize later how things will really need to be put together so you make alterations/refactor, you get feedback from QA testers/analysts/focus groups, and continually refine until you get the final product.

So unless you’re solving the exact same problem every time, what exactly are we basing the effort on? If a carpet guy knows he can carpet a 2 bedroom house in a day, he can be very close with an estimate on a 4 bedroom house. And overtime, the variation of houses is fairly limited.

However, every software project is a new problem; why would the business ask to solve the same problem twice? So we’re estimating based off of as-similar projects done in the past, but unless it is purely a CRUD with zero rules, it’s unique.

Of course an estimate will always be an estimate. But plans & dates get created, promises are made around those estimates, and the estimates become commitments. Which then causes developers to factor in some fudge knowing that the estimate will become a commitment.

Would it help if we get rid of the word estimate and label that value for what it really is, the guess? Imagine updating all your change tracking, ticketing systems, and project management tools to replace the “Estimate” field with the term “Guess”? It would help manage some expectations as to what that number really represents.

The flip side

The problem on the flip side is that businesses are run by plans. They’re not designed to function off of guesses. For example Apple has to gets the contracts signed and dates sealed for the Apple WWDC, payments have to be made to Moscone, Moscone has to prep the building, Apple has to build their booths, sign up all the exhibitors, attendees, etc… so whatever software is planned to be presented has to be ready to roll by then.

It’s a fascinating challenge, and here are some thoughts.

1. Break down the problem into a collection of micro-small problems.
• Smaller problems are easier to size, internalize, and visualize. Many Agile based processes employ this in the form of small as possible user stories.
2. Get formulaic about it.
• If a team consistently applies a 2X b.s. factor, and is 80% accurate – you then have variables to build a formula & pattern around it where you project a target date -/+ x% based on the accuracy, and then build in risk mitigation strategies to make sure you’re managing the risk of not being accurate.
3. Prioritize & focus.
• A good practice at any level (from executive to individual contributor) at any thing (business, coding, personal life).
• Make sure you focus on achieving at least the top priority in it’s entirety by the target date instead of having advanced multiple priorities but never fully completing any. One thing done is better then 10 things almost-done, because almost-done is the same at not done.

One of the problems with the traditional waterfall method is that it’s predicated on the notion that it’s possible with enough planning and requirements gathering to accurately estimate out the effort of a project.

If that were even possible, and your life depended on it, you wouldn’t feel comfortable until the team has spent an enormous amount of time putting together analysis and various levels of technical design.

In order for that possibility to exist you would have to be aiming for a fixed target, whereas developing software is a moving target where scope can change at an time (agile embraces and expects change).

So after weeks or even months of analysis you drop the bomb on the stakeholders of the project that what was asked for won’t fit the required timeline without a decrease in scope or change another project dimension (capacity, budget, resources, etc…).

Meanwhile, stakeholders want an estimation done immediately. There may be a constraint or deadline that needs to be hit, there may be a funding window to get approval, stakeholders might need time lining up shared resources and don’t want to reserve people if they’re going to end up vegging out on the bench waiting for the project to commence, etc…

So what’s the compromise? Story points!

Story points solve the problem

The problem is we need a quick and easy way to schedule out work. In the waterfall approach the business views estimates as commitments, but estimates are estimates – they’re not actuals. You’ll only ever know the actual once the work is completed.

In waterfall land your plan relies on as accurate as possible estimates because you build out your timeline based on that. With agile, your work is based off of fixed-length time boxes – so you know how long each iteration is going to be, the only question is what you fit inside each of those fixed sized buckets of time.

So imagine you’re filling up boxes with various household items, do you really need to know down to millimeter accuracy how big each object is in order to have a sense of how much you can fit in a box? More importantly would it be worth it to spend 2 weeks cataloging all your household items into Excel using a measuring tape in order to make a plan that fills each box to it’s maximum capacity – or is there more value in spending that time to just start filling the boxes by eyeballing roughly what will fit?

I know that some things are twice as big as others, I may not know their exact dimensions but I can guestimate that I can probably fit:

• 10 small things
• 5 small things, and 2 big things
• 4 small things, 2 medium things, and 1 big thing

In this case I’m working on filling a box, but on another project I could be working on buildings and I may not even need to change my point system much.

My buildings might fit into a 1 (small) to 5 (really big) range of categorization. The nice thing about using points is you get away from using time to estimate size. In theory you could use hours, such as 1hr for small things, 5hrs for medium, and 10hrs for large. But the problem is that when you start talking hours people start assuming you’re talking about duration or effort, so it’s best to use points or any term that signifies size.

Story points converted to time

These numbers initially can’t be converted into time, you have no data in order to base such a conversion on. But as you perform iterations over time you’ll be able to track how many story points are completed with each iteration and thus know what your velocity is. Knowing that an iteration was 60 man hours, and it was 6 story points, you could formulate that a story point is about 10hrs of effort.

Knowing actual hours might be an interesting tidbit, but one could argue they’re irrelevant. This is because when you’re planning out your next iteration you’re still using the story point technique to fill your boxes, but now you have historical data to improve your understanding of how many points fit into a box.

Recap

• Estimates are estimates, not actuals.
• Actuals are historical, you can never know them upfront.
• No developer in their right mind would commit to an estimate without heavy duty specs.
• Waterfall requires time consuming analysis in order to get specifications detailed enough for someone to commit effort and duration to. Knowing effort, waterfall then plans out the timeline.
• Agile is fixed-length timelines. You already know when an iteration/phase/sprint ends. You just need to fill up those iterations with work.
• Story points are a quick way to estimate out the relative size of one feature vs. another feature. Size is the keyword (vs. effort or duration).
• Using relative sizes, you’ll have a rough idea of how many points can be completed within an iteration.
• As iterations end, you’ll have a stronger understanding of how many points an iteration can fit.