I propose a different approach to software development estimation.
The first question to be asked should be:
1) When can you start development on feature X?
This question should lead to discussion and not a simple answer like "In two weeks". If you get a simple answer then ask, "Why then?" This will cause an explanation of what has to be done before the new feature can be started. This will expose the developer's priorities of work and will allow new priorities to be considered.
The second question immediately follows the discussion of the first question and is made up of two parts (so you might want to say it is really two questions).
2) Do you feel that feature X is complex and have you ever done anything like feature X before?
Notice there is no estimate of how long the development of the new feature will take. This is key. Also notice there is an estimate of what the developer is currently doing. This is exactly what is wanted. By saying when one can start working of feature X it requires one to estimate when they will finish with feature W. The estimation for completion of feature W happens after feature W has been started and thus the estimation is based on experience up to that point! I think that is great.
This brings us back to the essential nature of question 2. If question two identifies the problem as being complex, unfamiliar, or both, then you should recognize that any estimates start out as complete guesses and as experience is gained the estimates take on the form of educated guesses.
So lets go through a little example.
New Feature W: Find all customers that have purchased a monthly subscription and have canceled that subscription within two months of purchase.
Manager Lynn: Pat, when can you start on feature W?
Developer Pat: Lynn, I can start on it in 3 days.
Lynn: What happens in three days that will allow you to start on W?
Pat: I finish up feature V. All I have left to do is hook up the list control, handle the selection event and it is done.
Lynn: Okay. You've done a good job on feature V. Thanks.
Pat: Sure. That's why I get paid the big bucks!
Lynn and Pat chuckle.
Lynn: Do you think feature W is complex and have you ever worked on anything like it before?
Pat: It doesn't seem complex, but I am not familiar with the databases involved with making the queries, so I have to learn where and how to get to the data. I am pretty good with SQL so I don't think that will be a problem.
Lynn: Okay, that's good to know. We'll talk more about feature W later. Thanks.
Lynn goes away and starts to think:
"I wonder where that data is? I wonder if Pat has rights to access the data? Maybe I should ask the IT department and the DB admins about this data."
I hope this simple contrivance helps you imagine how this method of estimation works.
How can this be applied to Product estimation? Well, if the development team is working on another product then question one applies in that, "When will they be done with Product A?"
Question two still applies in that "Which aspects of product B are new and unfamiliar in concept and which aspects appear to be difficult or complex?"
I hope this "new" way of estimating software helps. This is part of what I call "Experience Driven Development". One day I will get that darn book finished!
You can do this method in conjunction with existing methods, that is, there is no rule that says you have to estimate using only one method! Use two, three, or four methods of estimation if you feel it is of value!
Sunday, December 09, 2007
Wednesday, November 14, 2007
Code Debt, Product Market Debt, and Customer Debt
During software development code debt is accrued when decisions adversely affect the ability to extend, expand, or maintain the software code base. It is hard for me to define code debt however I know when I am accruing code debt. For instance, if I copy and paste some code to create a new class instead of taking the time to figure out if there should be some base classes, virtual methods, or interfaces I know that I am accruing code debt. Often the choice to accrue code debt is based on expediency, stealing from the future to pay for today.
Product Market Debt is accruing anytime a competitor has their product in the market and your's is not. It is also accruing if a competitor has a better product.
Customer Debt accrues if your product is of poor quality, is missing essential features, is hard to use, or any other aspect that the customer finds less than satisfactory.
Before the first release of the product the Code Debt is only known to the developers. The product marketing team will not recognize code debt and will not understand the concerns of the developers. Clearly the customers will not see any of the effects of code debt for a product that has not released!
Before the first release of the product the Product Market Debt is the driving concern of the product marketing team. Often the developers do not feel the same sense of urgency that the product market team feels. Again the customers will not know of any Product Market Debt for a product that hasn't sipped.
In order to avoid the accrual of too much Product Market Debt the product marketing team will set a release date. Product marketing knows that Customer Debt will be accrued if there are essential features missing and therefore product marketing creates a list of features to be released.
The feature list and the release date become a concern for the developers. Development knows that if they try to do too much in too short of a time that they will accrue Code Debt. The developers know that too much code debt will result in Customer Debt followed immediately by the accrual of Product Market Debt. For example if the Code Debt is high and the product is unreliable the customer will immediately become dissatisfied with the product. Time and money spent to increase the product reliability is the monetary value of Code Debt. While the developers are spending their time fixing bugs to increase reliability they are NOT developing new features and thus we are now accruing Product Market Debt because essentially we are falling behind. Loss of market share can be calculated as a monetary value which is the value of Product Market Debt. Customers returning the product and the lack or loss of sales is the monetary value associated with Customer Debt.
The interaction of each of these Debts and the effects of Debt weigh heavily on the company's decision making process. Often the group that has the most influence within the company will control which type of debt is the primary concern. Often if the company was founded on a software product and grew around this product the concern of debt will be Code Debt, Customer Debt, and finally Product Market Debt. This may help one understand why some software companies fail in the market place. If the company is not a software company and is a company that is developing software to assist in its operations then the company may focus on Product Market Debt, Customer Debt, and finally Code Debt. This may help one understand why such companies struggle with software development cost overruns, continuous emergencies and outages of the software, and conflicts between the developers, the I.T. department, and the "Business".
Each type of Debt is constantly accruing. The plan is to manage the rate of accrual. Code Debt accrues even if you have the best code that has ever been developed. It accrues because the eventually re-write is always on its way. If your product is developed with .NET 2 for Windows XP eventually you will find that the Product Market Debt is sufficient to force a re-write for .NET 3 and Vista. The previous statement shows that Product Market Debt is always accruing as well such as the release of a competing product or a new development framework and a new computer operating system. Customer debt is always accruing as well. The customer sees new features of a new operating system and they want the product to take advantage of them or the customer sees a new technology and imagines how great it would be if the product integrated with that technology.
Either of the three Debts can break your company.
The position of the company influences which Debt will be of primary concern.
If the company is creating a new software product for an existing space then Product Market Debt may be the greatest concern. You have to get the product out to compete with the existing products and hopefully you can leap ahead of the existing feature set. Underestimating the scope of the software development task often leads such a company to under staff development and thus create an environment that will heavily accrue Code Debt. I advise that you should remember that all three Debts are accruing for the competitors as well. If you are fortunate enough to plan the development of your product coinciding with the release of a new operating system (or other Product Market Debt driving force) then the competitors will be obliged to pay for their Product Market Debt and overhaul or maybe even re-write their product.
If the company has a mature product with a loyal customer base then the company may choose to accrue more Code Debt and hurry out a new release to ward off a new competitor or a new release of an existing competitor's product. The accrual of the Code Debt will result in an increased rate of accrual for Customer Debt. The Code Debt will result in a lower quality product and thus the customer will be dissatisfied. If the goal is to quickly release software updates/patches to the customer to pay off the Customer Debt you will find you have fallen into a trap. There is no quick release when there is Code Debt. The need for the quick release will cause decisions such as "throwing more people" at the problem or accruing more Code Debt.
If the company is developing a new software product for a new market space then there is no Customer Debt as of yet. Product Market Debt is hard to measure because you do not know if another company has had the same idea and is about to release. Code Debt does not exist yet because there is no code, however the first debt that will start to accrue is Code Debt, and it starts as soon as the first line of code is wrote.
Sufficient funding is one aspect of controlling all three Debts. However it is not sufficient to guarantee success. If you have sufficient funding to hire enough product marketing people to define the product and the needed release schedule, and enough developers to create the product with all of the listed features by the release date with the highest quality code possible and enough customer interaction during development then all three Debts have been addressed. However the money is not sufficient to guarantee success because you may have hired enough developers, enough product marketers, and other persons necessary but if these people are inefficient or incompetent all of the money in the world will not "save you".
Product Market Debt is accruing anytime a competitor has their product in the market and your's is not. It is also accruing if a competitor has a better product.
Customer Debt accrues if your product is of poor quality, is missing essential features, is hard to use, or any other aspect that the customer finds less than satisfactory.
Before the first release of the product the Code Debt is only known to the developers. The product marketing team will not recognize code debt and will not understand the concerns of the developers. Clearly the customers will not see any of the effects of code debt for a product that has not released!
Before the first release of the product the Product Market Debt is the driving concern of the product marketing team. Often the developers do not feel the same sense of urgency that the product market team feels. Again the customers will not know of any Product Market Debt for a product that hasn't sipped.
In order to avoid the accrual of too much Product Market Debt the product marketing team will set a release date. Product marketing knows that Customer Debt will be accrued if there are essential features missing and therefore product marketing creates a list of features to be released.
The feature list and the release date become a concern for the developers. Development knows that if they try to do too much in too short of a time that they will accrue Code Debt. The developers know that too much code debt will result in Customer Debt followed immediately by the accrual of Product Market Debt. For example if the Code Debt is high and the product is unreliable the customer will immediately become dissatisfied with the product. Time and money spent to increase the product reliability is the monetary value of Code Debt. While the developers are spending their time fixing bugs to increase reliability they are NOT developing new features and thus we are now accruing Product Market Debt because essentially we are falling behind. Loss of market share can be calculated as a monetary value which is the value of Product Market Debt. Customers returning the product and the lack or loss of sales is the monetary value associated with Customer Debt.
The interaction of each of these Debts and the effects of Debt weigh heavily on the company's decision making process. Often the group that has the most influence within the company will control which type of debt is the primary concern. Often if the company was founded on a software product and grew around this product the concern of debt will be Code Debt, Customer Debt, and finally Product Market Debt. This may help one understand why some software companies fail in the market place. If the company is not a software company and is a company that is developing software to assist in its operations then the company may focus on Product Market Debt, Customer Debt, and finally Code Debt. This may help one understand why such companies struggle with software development cost overruns, continuous emergencies and outages of the software, and conflicts between the developers, the I.T. department, and the "Business".
Each type of Debt is constantly accruing. The plan is to manage the rate of accrual. Code Debt accrues even if you have the best code that has ever been developed. It accrues because the eventually re-write is always on its way. If your product is developed with .NET 2 for Windows XP eventually you will find that the Product Market Debt is sufficient to force a re-write for .NET 3 and Vista. The previous statement shows that Product Market Debt is always accruing as well such as the release of a competing product or a new development framework and a new computer operating system. Customer debt is always accruing as well. The customer sees new features of a new operating system and they want the product to take advantage of them or the customer sees a new technology and imagines how great it would be if the product integrated with that technology.
Either of the three Debts can break your company.
The position of the company influences which Debt will be of primary concern.
If the company is creating a new software product for an existing space then Product Market Debt may be the greatest concern. You have to get the product out to compete with the existing products and hopefully you can leap ahead of the existing feature set. Underestimating the scope of the software development task often leads such a company to under staff development and thus create an environment that will heavily accrue Code Debt. I advise that you should remember that all three Debts are accruing for the competitors as well. If you are fortunate enough to plan the development of your product coinciding with the release of a new operating system (or other Product Market Debt driving force) then the competitors will be obliged to pay for their Product Market Debt and overhaul or maybe even re-write their product.
If the company has a mature product with a loyal customer base then the company may choose to accrue more Code Debt and hurry out a new release to ward off a new competitor or a new release of an existing competitor's product. The accrual of the Code Debt will result in an increased rate of accrual for Customer Debt. The Code Debt will result in a lower quality product and thus the customer will be dissatisfied. If the goal is to quickly release software updates/patches to the customer to pay off the Customer Debt you will find you have fallen into a trap. There is no quick release when there is Code Debt. The need for the quick release will cause decisions such as "throwing more people" at the problem or accruing more Code Debt.
If the company is developing a new software product for a new market space then there is no Customer Debt as of yet. Product Market Debt is hard to measure because you do not know if another company has had the same idea and is about to release. Code Debt does not exist yet because there is no code, however the first debt that will start to accrue is Code Debt, and it starts as soon as the first line of code is wrote.
Sufficient funding is one aspect of controlling all three Debts. However it is not sufficient to guarantee success. If you have sufficient funding to hire enough product marketing people to define the product and the needed release schedule, and enough developers to create the product with all of the listed features by the release date with the highest quality code possible and enough customer interaction during development then all three Debts have been addressed. However the money is not sufficient to guarantee success because you may have hired enough developers, enough product marketers, and other persons necessary but if these people are inefficient or incompetent all of the money in the world will not "save you".
Thursday, November 08, 2007
Process Rockstar
I'm through working with people
that never listen
It's like the bottom of the ninth
And I'm never gonna win
This life hasn't turned out
Quite the way I want it to be
(Tell me what you want)
I want a brand new process
With new buzz words
I want a write a book
for all those nerds
And have all my friends rate it on Amazon for me
(So what you need?)
I need an internet connection that's got no limit
And a new users group where I can spin it
Gonna sign the manifesto and kiss everybody's feet
(Been there, done that)
I want a new certification
that I control
My name mentioned
at conference shows
Somewhere between Brookes
and Gamma is fine with me.
(So how you gonna do it?)
I'm gonna trade programming for fortune and fame
I'll even do the iteration planninng game
'Cause we all just wanna be big Rock Stars....
that never listen
It's like the bottom of the ninth
And I'm never gonna win
This life hasn't turned out
Quite the way I want it to be
(Tell me what you want)
I want a brand new process
With new buzz words
I want a write a book
for all those nerds
And have all my friends rate it on Amazon for me
(So what you need?)
I need an internet connection that's got no limit
And a new users group where I can spin it
Gonna sign the manifesto and kiss everybody's feet
(Been there, done that)
I want a new certification
that I control
My name mentioned
at conference shows
Somewhere between Brookes
and Gamma is fine with me.
(So how you gonna do it?)
I'm gonna trade programming for fortune and fame
I'll even do the iteration planninng game
'Cause we all just wanna be big Rock Stars....
Tuesday, October 23, 2007
Database "Design By Use"
About two years ago I designed a database for managing customer information, billing information, product information, pricing information, shipping information, and subscription information. The method I used to design the database, tables, and queries was Design By Use (DBU).
I can not share the resulting system but I can tell you how DBU helped with the design process.
The first step is to list as many of the "requests" you can define at the time. For example:
Find all of the customers who purchased product X during the past 6 months.
Find all of the products for sale in Spain.
Find all of the monthly subscriptions that have expired.
These examples are how the system will be used, thus we are designing the system in the manner we want to use the system.
The above examples eventually became SQL queries.
At the same time the usage examples are helping formulate the design the domain objects are being identified as well. The domain objects eventually found their representation in the database tables either as a table schema or as the results of joins on various tables.
Knowing how the database was to be used also helped in figuring out the schema and which tables might need keys to be used in other tables.
In the end I had designed a system that was exactly what was needed, thus YAGNI was satisfied. Also it was demonstrable in that the queries could be ran against test data and through automated testing show that the system correctly works.
I have designed many databases over the years but this approach was by far the best. It was better because it did exactly what was needed, no more, no less, and that I could run the queries as regression tests.
Often I have dealt with cross database joins in other systems and often I have found that such activities are the results of poor design.
Design By Use works great for me. You should try it and see if you find it helpful as well!
I can not share the resulting system but I can tell you how DBU helped with the design process.
The first step is to list as many of the "requests" you can define at the time. For example:
Find all of the customers who purchased product X during the past 6 months.
Find all of the products for sale in Spain.
Find all of the monthly subscriptions that have expired.
These examples are how the system will be used, thus we are designing the system in the manner we want to use the system.
The above examples eventually became SQL queries.
At the same time the usage examples are helping formulate the design the domain objects are being identified as well. The domain objects eventually found their representation in the database tables either as a table schema or as the results of joins on various tables.
Knowing how the database was to be used also helped in figuring out the schema and which tables might need keys to be used in other tables.
In the end I had designed a system that was exactly what was needed, thus YAGNI was satisfied. Also it was demonstrable in that the queries could be ran against test data and through automated testing show that the system correctly works.
I have designed many databases over the years but this approach was by far the best. It was better because it did exactly what was needed, no more, no less, and that I could run the queries as regression tests.
Often I have dealt with cross database joins in other systems and often I have found that such activities are the results of poor design.
Design By Use works great for me. You should try it and see if you find it helpful as well!
Saturday, September 08, 2007
Maverick Web Site
Quick note: The maverick web site has been moved. The new URL is:
http://home.comcast.net/~gslinker/agile.html
http://home.comcast.net/~gslinker/agile.html
Thursday, June 07, 2007
What is Software?
Often I have heard people try to describe software. If it can be described then maybe we can formalize it and then make the process of software development more like a true engineering process.
I propose that one should think of software like the blue prints of a house instead of thinking of software like components, objects, and building parts. Let me try to explain. I do not think that when a software developer is writing code he is building the metaphoric walls, floors, and roof of the house.
Before I go any further I want to stop and say I am not saying modules, objects, and other abstraction and organization techniques should not be used.
Software is a set of instructions that will be executed by a computer. These instructions are created in a manner that is suitable for humans to create, organize, and maintain but they are still just instructions. These instructions are turned over the the computer for processing. So if we go back to the house building analogy then the software engineer creates a set of instructions or blueprints on how to build the house and the computer's processor actually builds the house. Each time the software is launched the proverbial house is being rebuilt.
To further the analogy it would be like delivering a blueprint of the house along with ordered sequential instructions to guide every step and every action of the house builders. It might go like this: "Worker pick up hammer. Worker drive nail." But wait, we didn't tell the worker to pick up a nail and we didn't even have the worker standing in front of the wall in which we wish to drive the nail.
The interesting thing about using the house building analogy it is almost like the house is built when it is needed to be used and then it is torn down and all of its resources are available to be used by someone else. So it is like all of the nails that we used to build the house are available to use to build something else when our house is not in use. Nails are a resource needed to build a house. Similarly a storage device such as a hard drive is needed to store the software. Resources are finite that is to say there are only so many nails available or so much disk space available.
Once a house is built it continues to consume resources such as water and electricity. Software after it is built consumes memory and use resources like the network or video card. If there is no electricity the "house" fails to provide such things as light or heat to the customer. If there is no more memory the software fails to perform desired functions.
Thinking of software in this way helps one understand why software fails even though it had performed fine for months. Software depends upon the resources available in the environment for which it exists. The availability of resources change and thus software that worked before now fails. How the software fails is described as a level of robustness. Robust software fails nicely. That is the programmer provided instructions to the computer to do something in case there isn't enough memory or the network connection is down. To go back to the house building analogy it would be like giving instructions to the builder "Get a 6 penny nail. If there are no 6 penny nails then get an 8 penny nail. If there are no 8 penny nails then signal your boss you will wait until there are some nails..."
Software is just a set of instructions to be performed by a computer processing unit. Software is not some carved out thing on a hard drive that leaps to the forefront when you click on it.
To me that is what software is.
I propose that one should think of software like the blue prints of a house instead of thinking of software like components, objects, and building parts. Let me try to explain. I do not think that when a software developer is writing code he is building the metaphoric walls, floors, and roof of the house.
Before I go any further I want to stop and say I am not saying modules, objects, and other abstraction and organization techniques should not be used.
Software is a set of instructions that will be executed by a computer. These instructions are created in a manner that is suitable for humans to create, organize, and maintain but they are still just instructions. These instructions are turned over the the computer for processing. So if we go back to the house building analogy then the software engineer creates a set of instructions or blueprints on how to build the house and the computer's processor actually builds the house. Each time the software is launched the proverbial house is being rebuilt.
To further the analogy it would be like delivering a blueprint of the house along with ordered sequential instructions to guide every step and every action of the house builders. It might go like this: "Worker pick up hammer. Worker drive nail." But wait, we didn't tell the worker to pick up a nail and we didn't even have the worker standing in front of the wall in which we wish to drive the nail.
The interesting thing about using the house building analogy it is almost like the house is built when it is needed to be used and then it is torn down and all of its resources are available to be used by someone else. So it is like all of the nails that we used to build the house are available to use to build something else when our house is not in use. Nails are a resource needed to build a house. Similarly a storage device such as a hard drive is needed to store the software. Resources are finite that is to say there are only so many nails available or so much disk space available.
Once a house is built it continues to consume resources such as water and electricity. Software after it is built consumes memory and use resources like the network or video card. If there is no electricity the "house" fails to provide such things as light or heat to the customer. If there is no more memory the software fails to perform desired functions.
Thinking of software in this way helps one understand why software fails even though it had performed fine for months. Software depends upon the resources available in the environment for which it exists. The availability of resources change and thus software that worked before now fails. How the software fails is described as a level of robustness. Robust software fails nicely. That is the programmer provided instructions to the computer to do something in case there isn't enough memory or the network connection is down. To go back to the house building analogy it would be like giving instructions to the builder "Get a 6 penny nail. If there are no 6 penny nails then get an 8 penny nail. If there are no 8 penny nails then signal your boss you will wait until there are some nails..."
Software is just a set of instructions to be performed by a computer processing unit. Software is not some carved out thing on a hard drive that leaps to the forefront when you click on it.
To me that is what software is.
Wednesday, April 18, 2007
Improving Software Estimates Part 3
In the first post entitled "Improving Software Estimates" I described two scenarios where the estimates were incorrect.
First let me digress and say an estimate is neither right or wrong. It is an estimate. When I say an estimate was incorrect I mean the estimate and the actual results varied. In the world I live in an estimate for when software will be finished is a prediction. Predictions for software delivery dates are used to schedule dependent tasks. So, when I use the term estimate I am using the common vernacular of the industry. I prefer prediction.
Back to the point of this post. The second failure scenario is that the developer was developing something new and had no prior experience that could be directly applied. When faced with the unknown then I rely on "method" to help me through.
The "method" I use for tackling the unknown is called "Software Scouting and Reconnaissance".
“Scouting” is the metaphor. During the exploration of the American frontier, scouts were sent out ahead of the company to determine the safest path through unknown and hostile territory. Through software “scouting missions” one can save time and money, and reduce the risks to the company.
When one is in unknown territory scout ahead for information and then come back and apply the knowledge gained. Have enough discipline not to get distracted by the sights along the way. Stay focused, travel light, and get back to camp as quickly as possible.
This pattern of short scouting trips would continually repeat, making the technique both iterative and incremental.
Before starting the journey into the unknown you must do as much study as possible. Gain as much knowledge as you can so that you can apply it as you go.
After the study is complete then define your domains and sub-domains. Create your dictionary of terms and metaphors that you will use to describe the potential solution. In other words I recommend coming up with a high level design. This design can be in your head, on paper with diagrams or text, what ever works for you.
This design is used to break the whole into smaller tasks. These smaller tasks are your weigh stations along the journey.
All of this is preparatory work so that you can take your first heading. Every journey starts in some direction.
At this point I start writing code. Starting with the end in mind is one of the best ways I know to get you going. One can analyze and study the problem for way too long especially if you are nervous about the first step. I use Design By Use (DBU) to do this. I write the code as if the methods already exist. It is very much like Test First Programming described in Extreme Programming. In Part 2 of this series I described DBU as a method to synchronize two teams. DBU can be applied even if you are going to write all of the code and you have no external dependencies.
"Code your way" towards the completion of the first small task. You will be learning as you go and now you can apply this knowledge and experience and revise your estimate of completion. You continue in this iterative and incremental fashion until the remaining task is so small that you feel you can't miss your estimate too badly or until you have enough experience that there is nothing unknown left and you can estimate the remaining work just like you would on a task where you have experience.
To recap when I am faced with developing software where I do not have any experience I rely on processes and methods to carry me forward. Specifically I use Software Scouting and Reconn and Design By Use to move forward in an incremental and iterative fashion until I have finished.
Geoff
First let me digress and say an estimate is neither right or wrong. It is an estimate. When I say an estimate was incorrect I mean the estimate and the actual results varied. In the world I live in an estimate for when software will be finished is a prediction. Predictions for software delivery dates are used to schedule dependent tasks. So, when I use the term estimate I am using the common vernacular of the industry. I prefer prediction.
Back to the point of this post. The second failure scenario is that the developer was developing something new and had no prior experience that could be directly applied. When faced with the unknown then I rely on "method" to help me through.
The "method" I use for tackling the unknown is called "Software Scouting and Reconnaissance".
“Scouting” is the metaphor. During the exploration of the American frontier, scouts were sent out ahead of the company to determine the safest path through unknown and hostile territory. Through software “scouting missions” one can save time and money, and reduce the risks to the company.
When one is in unknown territory scout ahead for information and then come back and apply the knowledge gained. Have enough discipline not to get distracted by the sights along the way. Stay focused, travel light, and get back to camp as quickly as possible.
This pattern of short scouting trips would continually repeat, making the technique both iterative and incremental.
Before starting the journey into the unknown you must do as much study as possible. Gain as much knowledge as you can so that you can apply it as you go.
After the study is complete then define your domains and sub-domains. Create your dictionary of terms and metaphors that you will use to describe the potential solution. In other words I recommend coming up with a high level design. This design can be in your head, on paper with diagrams or text, what ever works for you.
This design is used to break the whole into smaller tasks. These smaller tasks are your weigh stations along the journey.
All of this is preparatory work so that you can take your first heading. Every journey starts in some direction.
At this point I start writing code. Starting with the end in mind is one of the best ways I know to get you going. One can analyze and study the problem for way too long especially if you are nervous about the first step. I use Design By Use (DBU) to do this. I write the code as if the methods already exist. It is very much like Test First Programming described in Extreme Programming. In Part 2 of this series I described DBU as a method to synchronize two teams. DBU can be applied even if you are going to write all of the code and you have no external dependencies.
"Code your way" towards the completion of the first small task. You will be learning as you go and now you can apply this knowledge and experience and revise your estimate of completion. You continue in this iterative and incremental fashion until the remaining task is so small that you feel you can't miss your estimate too badly or until you have enough experience that there is nothing unknown left and you can estimate the remaining work just like you would on a task where you have experience.
To recap when I am faced with developing software where I do not have any experience I rely on processes and methods to carry me forward. Specifically I use Software Scouting and Reconn and Design By Use to move forward in an incremental and iterative fashion until I have finished.
Geoff
Sunday, April 15, 2007
Improving Software Estimates Part 2
In the first post I described two scenarios where the estimate was incorrect. In the first description the delivery estimate was incorrect because of external dependencies.
Software is often developed by teams of engineers. Also it is common that several teams work simultaneously. Estimation is important when coordinating several teams.
I have addressed the issue of team coordination with a process known as Design By Use (DBU). DBU is not intended to be the single solution to coordination issues but is intended as a tool.
In the scenario previously described the developer was unable to meet his estimated delivery schedule because of an external dependency. This is often a "down stream" dependency. You can not continue until the other team has delivered so that you can link to their system. DBU addresses this issue by defining the integration interface early and integrating immediately.
When designing code the "caller" has certain data available at specific levels of the system. The "callee" does not know what data is available to the caller and therefore it is risky for the callee to define the interface based on assumptions. Assumptions of data availability may break domain models and layers of separation. Therefore DBU teaches us that the caller should write a usage example and give it to the callee. Do you see how this addresses the coupling concerns had by the caller?
Also the callee is benefitted by DBU. The principal of "you aint going to need it" (YAGNI) is addressed by DBU. I have written many internal utility libraries over the years. Without knowing exactly how your library is to be used by other teams at your company it is easy to add features and overloaded methods in the attempt to "be everything for everybody". This ultimately leads to lots of code that is never called and which become maintenance baggage from "then on out".
By addressing YAGNI you gain the benefits of improved efficiency in that you are not writing anything extra. You only write what someone is going to use and you know that they are going to use it because they (the caller) specified the interface.
How is software estimation improved?
The caller controls their domain and abstraction layers and protects the coupling of the system. Loosely coupled systems are easier to work with and I propose that systems that are loosely coupled assists in the estimation process. If the caller discovers that additional systems must be coupled it is known sooner rather than later. This addresses integration issues which have proven to be one of the most costly activities of software development if ignored until the end.
The callee is also helped with the estimation task by not having to design a system that will "be everything to everybody". They can focus on the specified interactions exclusively. The first thing they can do is provide a stubbed version of the system to the caller so that they can immediately link to their system. Then the caller can continue to work while the callee implements the solution. Pretty slick I do have to say.
Here is a link to a specific application of DBU. It describes how DBU can be used to organize multiple teams.
http://home.comcast.net/~gslinker/maverick/DBU.html
If you address integration issues early and only write what is needed software estimates can be made simpler and it is my belief that reduced complexity is directly related to improved software estimation.
Software is often developed by teams of engineers. Also it is common that several teams work simultaneously. Estimation is important when coordinating several teams.
I have addressed the issue of team coordination with a process known as Design By Use (DBU). DBU is not intended to be the single solution to coordination issues but is intended as a tool.
In the scenario previously described the developer was unable to meet his estimated delivery schedule because of an external dependency. This is often a "down stream" dependency. You can not continue until the other team has delivered so that you can link to their system. DBU addresses this issue by defining the integration interface early and integrating immediately.
When designing code the "caller" has certain data available at specific levels of the system. The "callee" does not know what data is available to the caller and therefore it is risky for the callee to define the interface based on assumptions. Assumptions of data availability may break domain models and layers of separation. Therefore DBU teaches us that the caller should write a usage example and give it to the callee. Do you see how this addresses the coupling concerns had by the caller?
Also the callee is benefitted by DBU. The principal of "you aint going to need it" (YAGNI) is addressed by DBU. I have written many internal utility libraries over the years. Without knowing exactly how your library is to be used by other teams at your company it is easy to add features and overloaded methods in the attempt to "be everything for everybody". This ultimately leads to lots of code that is never called and which become maintenance baggage from "then on out".
By addressing YAGNI you gain the benefits of improved efficiency in that you are not writing anything extra. You only write what someone is going to use and you know that they are going to use it because they (the caller) specified the interface.
How is software estimation improved?
The caller controls their domain and abstraction layers and protects the coupling of the system. Loosely coupled systems are easier to work with and I propose that systems that are loosely coupled assists in the estimation process. If the caller discovers that additional systems must be coupled it is known sooner rather than later. This addresses integration issues which have proven to be one of the most costly activities of software development if ignored until the end.
The callee is also helped with the estimation task by not having to design a system that will "be everything to everybody". They can focus on the specified interactions exclusively. The first thing they can do is provide a stubbed version of the system to the caller so that they can immediately link to their system. Then the caller can continue to work while the callee implements the solution. Pretty slick I do have to say.
Here is a link to a specific application of DBU. It describes how DBU can be used to organize multiple teams.
http://home.comcast.net/~gslinker/maverick/DBU.html
If you address integration issues early and only write what is needed software estimates can be made simpler and it is my belief that reduced complexity is directly related to improved software estimation.
Saturday, April 14, 2007
Improving Software Estimates
As I work on my book, Experience Driven Development, I am compelled to share thoughts.
An important aspect of software development is the estimation of when software features will be complete. I have heard it said that the more you estimate the better you get at it. Is it true?
The estimate for "work" may be different than the estimate for "delivery".
Someone new at estimation may say "I will have the task done in 80 hours". While working on the task he soon finds out that a dependency is not ready and thus he has to wait. Suppose the delay is 40 hours so in the end it took 120 hours to complete the task. It is possible that if you only consider the person's time spent on the task it was 80 hours so their estimate of work was accurate. However the estimate of delivery was inaccurate because he spent 40 hours waiting on a dependency.
I believe that the experience will help the person make a better estimate next time. A better estimate in that it is more complete in considering dependencies.
Now I will describe another scenario. Suppose someone estimates that their feature will be done in 80 hours. There are no external dependencies. In reality suppose it took 100 hours to finish the feature. In this scenario the reason it took longer than estimated was because it was a new task and the person had never done anything quite like it before. In my opinion this kind of estimate will not improve. I do not think you can get better at predicting the unknown. What has happened is that the person has gained new experience and if that person encounters a similar situation in the future then he can estimate how long it will take to develop the familiar "parts" and add some amount of a guess to the estimate for the unfamiliar parts.
Experience improves estimates.
An important aspect of software development is the estimation of when software features will be complete. I have heard it said that the more you estimate the better you get at it. Is it true?
The estimate for "work" may be different than the estimate for "delivery".
Someone new at estimation may say "I will have the task done in 80 hours". While working on the task he soon finds out that a dependency is not ready and thus he has to wait. Suppose the delay is 40 hours so in the end it took 120 hours to complete the task. It is possible that if you only consider the person's time spent on the task it was 80 hours so their estimate of work was accurate. However the estimate of delivery was inaccurate because he spent 40 hours waiting on a dependency.
I believe that the experience will help the person make a better estimate next time. A better estimate in that it is more complete in considering dependencies.
Now I will describe another scenario. Suppose someone estimates that their feature will be done in 80 hours. There are no external dependencies. In reality suppose it took 100 hours to finish the feature. In this scenario the reason it took longer than estimated was because it was a new task and the person had never done anything quite like it before. In my opinion this kind of estimate will not improve. I do not think you can get better at predicting the unknown. What has happened is that the person has gained new experience and if that person encounters a similar situation in the future then he can estimate how long it will take to develop the familiar "parts" and add some amount of a guess to the estimate for the unfamiliar parts.
Experience improves estimates.
Subscribe to:
Posts (Atom)