Estimating Software Releases

The completion of a software development task without a risk estimate is a lie.

Estimating the completion of software development tasks is one of the more difficult parts of the job. Estimates are really, really, important. First, since someone else, either a customer or another developer, is waiting for your code, they need to know when it will be available. Secondly, the estimation process will help you identify critical risk tasks. These are tasks that can make your software miss it's mark. The software estimate is the equivalent of an outline for a paper -it's a mini simulation that gets everyone on the same page. Lastly, estimates are a key component to your ability to set and manage expectations for customers and stakeholders.

There are many things that get in the way of a timely release. They include:

• Lack of definition ( can't estimate it if you don't know what it is)


• Feature creep (no estimate existed)


• Underestimation ( too optimistic )


• How long is a piece of string issues (impossible to estimate w/ accuracy)


• Things you didn't estimate (discovered work)


• Unexpected events (impossible to estimate w/ accuracy)

The first three are mistakes based on a lack of experience. If you spend enough time writing code, you will lean not to underestimate how much work it takes to develop commercial software and that constraining scope is important. Feature creep is usually a self-disipline issue.

The remaining issues are more often than not the reason why a well lead software team doesn't meet a deadline. So how do we address those issues?

There are many approaches. An early approach I used is part of the PERT method -a project estimation process that has it's roots 1950s rocket science. Simply put, for any given task an expected time is calculated using your best case (BC) guess, worst case(WC) guess, and most likely guess (ML). Using these terms, the expected time is:

Expected Time = ( BC + 4ML + WC )/6

What's the difference between this number and the most likely guess? The answer is risk.

For example, let's say you move from one company to another and they ask you to do a task you've done before. We will call that task job A. If that task took you two weeks, you should be fairly confident in your estimate of two weeks. So your estimate for job A is two weeks with a high degree of confidence.

Now, let's say that you are asked for another estimate for a similar job B. It's not exactly the same job, but using your past experience you estimate two weeks again.

Job A's estimate does not equal Job B's estimate. Again, the reason is risk.

Taking risk into account, we could really suggest the estimate should look like the following:

JobA Estimate+ Risk = Expected Time
JobB Estimate+ Risk = Expected Time

Since Risk in the first equation is very low, maybe even zero, JobA ~= Expected Time. But since the risk in JobB is high, the expected time is a range [JobB Estimate, JobB Estimate + Risk ].

I use the following formula to determine risk adjusted time:

(((10-Confidence)/9)*OriginalEstimate)+OriginalEstimate = RiskAdjustedEstimate

Here's the catch, you set expectations and manage resource allocation by the risk adjusted estimate, but the software developers need to aim for the original estimate. Otherwise, it's not risk management, it's an opportunity for feature creep.

As a developer, I appreciate a schedule that accurately reflects risk and sets me up to succeed with being on time. It's discouraging to always being late or always working extra hours to account for the risk. It's also nice to know that the risk padding is based on something real, not just arbitrary padding of "well, you know, software is always late". That thinking undermines the disipline required to stay on schedule.

Confidence Scale (this help sets a common confidence level across your company)
1 no clue - don't make decisions on this..we need to talk more
2 hardly a clue - don't make decisions on this..we need to talk more
3 someone else has done this and i read about it; job not well defined - don't make decisions on this..we need to talk more
4 someone else has done this and i think i understand this; job might not be well defined - don't make decisions on this..we need to talk more
5 done something similar to this before and this relates to that work - this estimate has a variance of +/- 50% of estimate
6 I think i understand this fairly well and I understand the goal -
7 the average case for programming when the requirements are understood -
8 a confident case for programming; average case for a lot of art work
9 I've done this before and it's very similar;
10 I'm probably not working on software...or...no matter what, I'm only going to work on this for a period of time ( i.e. 1 week of QA ).

**********************

So this takes me back to my original statement, a software estimate without a value for risk is a lie. It's just not accurate if the risk is anything greater than zero ( using the method above ).

If you've read this far, you might agree with me. And you may react the same way I did by jumping on the PERT bandwagon. I found that the PERT model was difficult when asking multiple people for estimates. I noticed quite a variance in worst case and best case estimates. So I found a simpler approach that seems to work quite well.

When I ask someone for an estimate, I ask them for an estimate and a confidence level. I then use the confidence level to generate a risk adjusted estimate. In my method, I tell someone that a confidence of 1 means that the the risk adjusted estimate is double of the estimate. If doubling doesn't sound like enough to represent the risk, then the estimator needs to decompose the problem further until it fits the model. This really makes us expose areas of risk to a greater degree.

By having people focus on one number, confidence, they get better over time. With the PERT model, there is no single value to focus on -which I believes makes it harder to refine your skill.

The confidence number is really useful. Not only is it useful for the task, it can also be helpful on an entire project level. After the estimate is done for the entire project, you can review the confidence of the estimate. A project confidence of 6 ( I like using average of all tasks ) or less is not very reliable. A project confidence of 7 or 8 is something you can watch more carefully but still plan on. A project confidence of 9 or 10 is fairly secure.

Now that you are armed with this knowledge, you may feel more confident about not slipping, but unfortunately, there is still more managing to do. Going back to original list of things that make a project slip let's take a look at the last three.

"How long is a piece of string issues" are ones that you cannot estimate. These are binary conditions of broken until it's not. Memory crashes often fall under this category. Until you reach the end of the string, you don't know how long it is. There's not a whole lot we can do other than spend money on things like Bounds Checker or other similar programs.

"Things you didn't estimate" happen as well. Rapid prototyping is a great way to reduce these problems. I've often said, it's not the things you estimate that get you, it's the things you didn't realize you needed to estimate. After I make an estimate, I like to pick out the low confidence tasks and try to rapid prototype or simulate those issues.

Lastly, unexpected events. These things happen. In fact, I'm writing this right now from a hospital bed recovering from an appendectomy. This was not in my project schedule. How do you plan for these...well..it's hard...but a reasonable entire project padding (known as sandbagging) for the "what if" case is reasonable. Sandbagging should be reasonable and based on the risk adjusted time.

Finally, if you are producing commerical software estimates ( like a project release ) you may choose to add feature creep to outpace your competition. This is a stratigic software slip that may work, but may backfire. This is yet another reason why you may not want to give your customers an expected release date.

So there you have it. If anyone ever gives you an estimate for completion of a software project and they don't give you a risk estimate -tell them to stop lying.