   |
 A few minutes ago, I uploaded a new free resource called Torque 2D. We've always been interested in Teachers Pay Teachers so we thought we would do a little experiment. You can find the product page here. It's just a 30 day trial, but more than that, it's a test to see if teachers are interested in bringing game development into their classroom. Game development is an interesting way to teach math & computer science in your classroom. We would be really interested to hear how you might use game development in your classroom. If you are interested in teaching game development, please feel free to contact me directly a ericp@garagegames.com. Your feedback will help us decide how we will continue on Teachers Pay Teachers. Regards, Eric Preisz - CEO of GarageGames http://www.garagegames.com P.S. Yes, that's a picture of my family on the left hand side. |
Monday, January 14, 2013
Teaching Game Development
Saturday, January 23, 2010
Estimating Software Releases
The completion of a software development task without a risk estimate is a lie.
Estimating the completion of software development tasks is one of the more difficult parts of the job. Estimates are really, really, important. First, since someone else, either a customer or another developer, is waiting for your code, they need to know when it will be available. Secondly, the estimation process will help you identify critical risk tasks. These are tasks that can make your software miss it's mark. The software estimate is the equivalent of an outline for a paper -it's a mini simulation that gets everyone on the same page. Lastly, estimates are a key component to your ability to set and manage expectations for customers and stakeholders.
There are many things that get in the way of a timely release. They include:
The remaining issues are more often than not the reason why a well lead software team doesn't meet a deadline. So how do we address those issues?
There are many approaches. An early approach I used is part of the PERT method -a project estimation process that has it's roots 1950s rocket science. Simply put, for any given task an expected time is calculated using your best case (BC) guess, worst case(WC) guess, and most likely guess (ML). Using these terms, the expected time is:
Expected Time = ( BC + 4ML + WC )/6
What's the difference between this number and the most likely guess? The answer is risk.
For example, let's say you move from one company to another and they ask you to do a task you've done before. We will call that task job A. If that task took you two weeks, you should be fairly confident in your estimate of two weeks. So your estimate for job A is two weeks with a high degree of confidence.
Now, let's say that you are asked for another estimate for a similar job B. It's not exactly the same job, but using your past experience you estimate two weeks again.
Job A's estimate does not equal Job B's estimate. Again, the reason is risk.
Taking risk into account, we could really suggest the estimate should look like the following:
JobA Estimate+ Risk = Expected Time
JobB Estimate+ Risk = Expected Time
Since Risk in the first equation is very low, maybe even zero, JobA ~= Expected Time. But since the risk in JobB is high, the expected time is a range [JobB Estimate, JobB Estimate + Risk ].
I use the following formula to determine risk adjusted time:
(((10-Confidence)/9)*OriginalEstimate)+OriginalEstimate = RiskAdjustedEstimate
Here's the catch, you set expectations and manage resource allocation by the risk adjusted estimate, but the software developers need to aim for the original estimate. Otherwise, it's not risk management, it's an opportunity for feature creep.
As a developer, I appreciate a schedule that accurately reflects risk and sets me up to succeed with being on time. It's discouraging to always being late or always working extra hours to account for the risk. It's also nice to know that the risk padding is based on something real, not just arbitrary padding of "well, you know, software is always late". That thinking undermines the disipline required to stay on schedule.
Confidence Scale (this help sets a common confidence level across your company)
1 no clue - don't make decisions on this..we need to talk more
2 hardly a clue - don't make decisions on this..we need to talk more
3 someone else has done this and i read about it; job not well defined - don't make decisions on this..we need to talk more
4 someone else has done this and i think i understand this; job might not be well defined - don't make decisions on this..we need to talk more
5 done something similar to this before and this relates to that work - this estimate has a variance of +/- 50% of estimate
6 I think i understand this fairly well and I understand the goal -
7 the average case for programming when the requirements are understood -
8 a confident case for programming; average case for a lot of art work
9 I've done this before and it's very similar;
10 I'm probably not working on software...or...no matter what, I'm only going to work on this for a period of time ( i.e. 1 week of QA ).
**********************
So this takes me back to my original statement, a software estimate without a value for risk is a lie. It's just not accurate if the risk is anything greater than zero ( using the method above ).
If you've read this far, you might agree with me. And you may react the same way I did by jumping on the PERT bandwagon. I found that the PERT model was difficult when asking multiple people for estimates. I noticed quite a variance in worst case and best case estimates. So I found a simpler approach that seems to work quite well.
When I ask someone for an estimate, I ask them for an estimate and a confidence level. I then use the confidence level to generate a risk adjusted estimate. In my method, I tell someone that a confidence of 1 means that the the risk adjusted estimate is double of the estimate. If doubling doesn't sound like enough to represent the risk, then the estimator needs to decompose the problem further until it fits the model. This really makes us expose areas of risk to a greater degree.
By having people focus on one number, confidence, they get better over time. With the PERT model, there is no single value to focus on -which I believes makes it harder to refine your skill.
The confidence number is really useful. Not only is it useful for the task, it can also be helpful on an entire project level. After the estimate is done for the entire project, you can review the confidence of the estimate. A project confidence of 6 ( I like using average of all tasks ) or less is not very reliable. A project confidence of 7 or 8 is something you can watch more carefully but still plan on. A project confidence of 9 or 10 is fairly secure.
Now that you are armed with this knowledge, you may feel more confident about not slipping, but unfortunately, there is still more managing to do. Going back to original list of things that make a project slip let's take a look at the last three.
"How long is a piece of string issues" are ones that you cannot estimate. These are binary conditions of broken until it's not. Memory crashes often fall under this category. Until you reach the end of the string, you don't know how long it is. There's not a whole lot we can do other than spend money on things like Bounds Checker or other similar programs.
"Things you didn't estimate" happen as well. Rapid prototyping is a great way to reduce these problems. I've often said, it's not the things you estimate that get you, it's the things you didn't realize you needed to estimate. After I make an estimate, I like to pick out the low confidence tasks and try to rapid prototype or simulate those issues.
Lastly, unexpected events. These things happen. In fact, I'm writing this right now from a hospital bed recovering from an appendectomy. This was not in my project schedule. How do you plan for these...well..it's hard...but a reasonable entire project padding (known as sandbagging) for the "what if" case is reasonable. Sandbagging should be reasonable and based on the risk adjusted time.
Finally, if you are producing commerical software estimates ( like a project release ) you may choose to add feature creep to outpace your competition. This is a stratigic software slip that may work, but may backfire. This is yet another reason why you may not want to give your customers an expected release date.
So there you have it. If anyone ever gives you an estimate for completion of a software project and they don't give you a risk estimate -tell them to stop lying.
Estimating the completion of software development tasks is one of the more difficult parts of the job. Estimates are really, really, important. First, since someone else, either a customer or another developer, is waiting for your code, they need to know when it will be available. Secondly, the estimation process will help you identify critical risk tasks. These are tasks that can make your software miss it's mark. The software estimate is the equivalent of an outline for a paper -it's a mini simulation that gets everyone on the same page. Lastly, estimates are a key component to your ability to set and manage expectations for customers and stakeholders.
There are many things that get in the way of a timely release. They include:
- Lack of definition ( can't estimate it if you don't know what it is)
- Feature creep (no estimate existed)
- Underestimation ( too optimistic )
- How long is a piece of string issues (impossible to estimate w/ accuracy)
- Things you didn't estimate (discovered work)
- Unexpected events (impossible to estimate w/ accuracy)
The remaining issues are more often than not the reason why a well lead software team doesn't meet a deadline. So how do we address those issues?
There are many approaches. An early approach I used is part of the PERT method -a project estimation process that has it's roots 1950s rocket science. Simply put, for any given task an expected time is calculated using your best case (BC) guess, worst case(WC) guess, and most likely guess (ML). Using these terms, the expected time is:
Expected Time = ( BC + 4ML + WC )/6
What's the difference between this number and the most likely guess? The answer is risk.
For example, let's say you move from one company to another and they ask you to do a task you've done before. We will call that task job A. If that task took you two weeks, you should be fairly confident in your estimate of two weeks. So your estimate for job A is two weeks with a high degree of confidence.
Now, let's say that you are asked for another estimate for a similar job B. It's not exactly the same job, but using your past experience you estimate two weeks again.
Job A's estimate does not equal Job B's estimate. Again, the reason is risk.
Taking risk into account, we could really suggest the estimate should look like the following:
JobA Estimate+ Risk = Expected Time
JobB Estimate+ Risk = Expected Time
Since Risk in the first equation is very low, maybe even zero, JobA ~= Expected Time. But since the risk in JobB is high, the expected time is a range [JobB Estimate, JobB Estimate + Risk ].
I use the following formula to determine risk adjusted time:
(((10-Confidence)/9)*OriginalEstimate)+OriginalEstimate = RiskAdjustedEstimate
Here's the catch, you set expectations and manage resource allocation by the risk adjusted estimate, but the software developers need to aim for the original estimate. Otherwise, it's not risk management, it's an opportunity for feature creep.
As a developer, I appreciate a schedule that accurately reflects risk and sets me up to succeed with being on time. It's discouraging to always being late or always working extra hours to account for the risk. It's also nice to know that the risk padding is based on something real, not just arbitrary padding of "well, you know, software is always late". That thinking undermines the disipline required to stay on schedule.
Confidence Scale (this help sets a common confidence level across your company)
1 no clue - don't make decisions on this..we need to talk more
2 hardly a clue - don't make decisions on this..we need to talk more
3 someone else has done this and i read about it; job not well defined - don't make decisions on this..we need to talk more
4 someone else has done this and i think i understand this; job might not be well defined - don't make decisions on this..we need to talk more
5 done something similar to this before and this relates to that work - this estimate has a variance of +/- 50% of estimate
6 I think i understand this fairly well and I understand the goal -
7 the average case for programming when the requirements are understood -
8 a confident case for programming; average case for a lot of art work
9 I've done this before and it's very similar;
10 I'm probably not working on software...or...no matter what, I'm only going to work on this for a period of time ( i.e. 1 week of QA ).
**********************
So this takes me back to my original statement, a software estimate without a value for risk is a lie. It's just not accurate if the risk is anything greater than zero ( using the method above ).
If you've read this far, you might agree with me. And you may react the same way I did by jumping on the PERT bandwagon. I found that the PERT model was difficult when asking multiple people for estimates. I noticed quite a variance in worst case and best case estimates. So I found a simpler approach that seems to work quite well.
When I ask someone for an estimate, I ask them for an estimate and a confidence level. I then use the confidence level to generate a risk adjusted estimate. In my method, I tell someone that a confidence of 1 means that the the risk adjusted estimate is double of the estimate. If doubling doesn't sound like enough to represent the risk, then the estimator needs to decompose the problem further until it fits the model. This really makes us expose areas of risk to a greater degree.
By having people focus on one number, confidence, they get better over time. With the PERT model, there is no single value to focus on -which I believes makes it harder to refine your skill.
The confidence number is really useful. Not only is it useful for the task, it can also be helpful on an entire project level. After the estimate is done for the entire project, you can review the confidence of the estimate. A project confidence of 6 ( I like using average of all tasks ) or less is not very reliable. A project confidence of 7 or 8 is something you can watch more carefully but still plan on. A project confidence of 9 or 10 is fairly secure.
Now that you are armed with this knowledge, you may feel more confident about not slipping, but unfortunately, there is still more managing to do. Going back to original list of things that make a project slip let's take a look at the last three.
"How long is a piece of string issues" are ones that you cannot estimate. These are binary conditions of broken until it's not. Memory crashes often fall under this category. Until you reach the end of the string, you don't know how long it is. There's not a whole lot we can do other than spend money on things like Bounds Checker or other similar programs.
"Things you didn't estimate" happen as well. Rapid prototyping is a great way to reduce these problems. I've often said, it's not the things you estimate that get you, it's the things you didn't realize you needed to estimate. After I make an estimate, I like to pick out the low confidence tasks and try to rapid prototype or simulate those issues.
Lastly, unexpected events. These things happen. In fact, I'm writing this right now from a hospital bed recovering from an appendectomy. This was not in my project schedule. How do you plan for these...well..it's hard...but a reasonable entire project padding (known as sandbagging) for the "what if" case is reasonable. Sandbagging should be reasonable and based on the risk adjusted time.
Finally, if you are producing commerical software estimates ( like a project release ) you may choose to add feature creep to outpace your competition. This is a stratigic software slip that may work, but may backfire. This is yet another reason why you may not want to give your customers an expected release date.
So there you have it. If anyone ever gives you an estimate for completion of a software project and they don't give you a risk estimate -tell them to stop lying.
Wednesday, January 13, 2010
My Fundemental Beliefs
Over the years, I've come up with a lot of sayings. These quips are culminations of the voices in my head. Before you judge me, let me explain. I think the voices come from somewhere near my conscience. Sometimes I don't listen to them and it always comes back to haunt me. Most of these voices are parts of my core beliefs and while they originate from my working life, I also find that these beliefs carry over into my personal life.
2) The key to happiness is expectation management - You can't always control what you can deliver, but you can control what someone expects. No one has ever been unhappy when their expectations were met or exceeded. When you try to set expectations, don't set them too low and over deliver. The difference needs to be subliminal, otherwise they will adjust thier own expectations next time.
3) Speak to others in their langauge, not yours - We all think and learn in different ways. Don't assume that what works for you works for everyone else. I'm a big fan of true colors. When I meet someone, I try and size up what "color" they are. I was a big skeptic at first, but I'm a believer now.
4) Be direct - don't be an a$$hole behind someone's back because then you are an a$$hole and a coward. When you aren't telling it to someone "straight", you are probably doing it for selfish reasons so that you don't "feel" like a bad person.
5) If something sounds really stupid, it's likely that only know part of the story - Most companies have very smart people, if something sounds dumb, there's probably more information that, when added to the percieved dumb decision, makes sense.
6) Good managers realize that their most important work is usually boring - face it, writing requirements and clear direction is boring. But when done correctly, it will really help someone get something done.
7) Good news can wait; bad news cannot - Fairly obvious here. But so often we do the opposite.
8) It's every manager's job to promote and grow their employees - there are many ways to success. Pick the path that best aligns with employee's career goals.
9) A poor unified vision is more likely to lead to success than several disjoint brillian visions - I've worked for companies that have too many "cooks in the kitchen".
10) Move the Rock! ( it means make progress..even if it's not right )
11) If you aren't two weeks ahead, you are two weeks behind - I've never worked on a project that couldn't use two more weeks of polish. If you are too weeks ahead, you are on schedule...don't take a break. It's also not a good time to add features...again...you are on schedule.
12 ) Lie to yourself - sometimes, this is the first step in doing anything you believe you can do.
13 ) You are too close to the problem - You can't understand the "Duh" factor ( see next ) when you have worked on something for too long. Empathy isn't achievable because you are too biased.
14) Listening to people is good, whatching what they do is better - This is in regard to useability testing. What they say is useful, but sometimes they don't realize what they need or understand what they are doing.
15) The "Duh" Factor ( borrowed from Paul Dana ) - it's a new standard for simplicity.
16) Argue with the facts, not the person - Why argue head to head with someone...get on the same side of the table and argue with the facts.
17) A small business is managed chaos - the sooner you accept this, the better you will get at managing chaos
18) Never hold someone else to a standard that you don't hold yourself to.
19) I may not be as smart as you...but I will work twice as hard as you.
20) My personal favorite: People respect you when you allow them to get more done more easily.
2) The key to happiness is expectation management - You can't always control what you can deliver, but you can control what someone expects. No one has ever been unhappy when their expectations were met or exceeded. When you try to set expectations, don't set them too low and over deliver. The difference needs to be subliminal, otherwise they will adjust thier own expectations next time.
3) Speak to others in their langauge, not yours - We all think and learn in different ways. Don't assume that what works for you works for everyone else. I'm a big fan of true colors. When I meet someone, I try and size up what "color" they are. I was a big skeptic at first, but I'm a believer now.
4) Be direct - don't be an a$$hole behind someone's back because then you are an a$$hole and a coward. When you aren't telling it to someone "straight", you are probably doing it for selfish reasons so that you don't "feel" like a bad person.
5) If something sounds really stupid, it's likely that only know part of the story - Most companies have very smart people, if something sounds dumb, there's probably more information that, when added to the percieved dumb decision, makes sense.
6) Good managers realize that their most important work is usually boring - face it, writing requirements and clear direction is boring. But when done correctly, it will really help someone get something done.
7) Good news can wait; bad news cannot - Fairly obvious here. But so often we do the opposite.
8) It's every manager's job to promote and grow their employees - there are many ways to success. Pick the path that best aligns with employee's career goals.
9) A poor unified vision is more likely to lead to success than several disjoint brillian visions - I've worked for companies that have too many "cooks in the kitchen".
10) Move the Rock! ( it means make progress..even if it's not right )
11) If you aren't two weeks ahead, you are two weeks behind - I've never worked on a project that couldn't use two more weeks of polish. If you are too weeks ahead, you are on schedule...don't take a break. It's also not a good time to add features...again...you are on schedule.
12 ) Lie to yourself - sometimes, this is the first step in doing anything you believe you can do.
13 ) You are too close to the problem - You can't understand the "Duh" factor ( see next ) when you have worked on something for too long. Empathy isn't achievable because you are too biased.
14) Listening to people is good, whatching what they do is better - This is in regard to useability testing. What they say is useful, but sometimes they don't realize what they need or understand what they are doing.
15) The "Duh" Factor ( borrowed from Paul Dana ) - it's a new standard for simplicity.
16) Argue with the facts, not the person - Why argue head to head with someone...get on the same side of the table and argue with the facts.
17) A small business is managed chaos - the sooner you accept this, the better you will get at managing chaos
18) Never hold someone else to a standard that you don't hold yourself to.
19) I may not be as smart as you...but I will work twice as hard as you.
20) My personal favorite: People respect you when you allow them to get more done more easily.
Subscribe to:
Posts (Atom)