Work in Progress (WIP) at the individual developer's level is the point of view for this blog post.
WIP for an individual developer (for this blog) is defined as any prioritized item in the backlog/work queue for the developer. The backlog works as any simple priority queue, take the next item with the highest priority. (Starvation of items is acceptable!)
First, a manufacturing analogy.
Suppose you are on an assembly line where you attach three widgets to a gadget. You take the gadget from a bin (your backlog) and line up "widget 1", insert screw, and tighten, then line up "widget 2", insert screw, and tighten, and finally line up "widget 3", insert screw, and tighten, and then place the widgetfied gadget into a completion bin (inventory).
Could you gain performance if the process was changed to this:
Take gadget from bin, line up "widget 1", insert screw, and tighten, place in bin. Do this for all gadgets with widget 1. Then take gadget with widget 1 from bin, line up "widget 2", insert screw, and tighten, and place in bin. Do this for each gadget until all of widget 2 attached and then do this for widget 3 and place results in inventory bin.
Of course the answer depends. It depends on many things.
If all three widgets use the same screw, and thus the same screw driver, then it might be faster to stay with the original approach.
If all three widgets use different screws then the following should be measured:
- How long does it take to switch screw drivers and select proper screw?
- How long does it take to reach into the bin and extract a gadget plus how long does it take to place a gadget into a completion bin?
- How long does it take to switch position to work with different screw driver, screws, and bin (backlog)?
With these measurement one can decide which approach is faster.
Back to software development.
If the developer's backlog contains work that is related then the developer doesn't need to retool. For instance, if the developer works on a specific feature set, the domain is well defined, and the developer works with a small number of tools.
If the developer has many areas of responsibility, front end, middle tier, database layer, web services, etc., the developer has to mentally retool to work in these different areas.
In this situation the I suggest the consideration of a multiple queues for the single developer. The prioritization process is more complex, but the waste of retooling could be addressed. It may not be possible, but it should be considered. I wouldn't go too crazy with this. The reason is that if the code is being developed like a spike then the domain knowledge and flow from top to bottom is continuous and is good.
Switching context on features is expensive as well. I personally find it more expensive than switch context for tools.
If the developer's queue is made up of features (that are not subdivided so finely that it says, make UI change, make middle tier change, and make database change) then the developer can work on the feature from start to finish, keeping the domain context fresh and active in his mind. I prefer this approach.
If two features are related, like making a horizontal hourglass chart and a vertical hourglass chart, then the only difference is layout. The developer should look at the items in his queue and take "make horizontal hourglass chart" and "make vertical hourglass chart" and change it to "make hourglass chart", "vertical layout for hourglass chart" and "horizontal layout for hourglass chart".
I have been studying the idea that limiting the number of items in the work queue (the WIP) can increase throughput. This is known as Limiting WIP. It is suggested that the time to manage items in the queue is costly and therefore if the items are limited then the cost to manage them is limited.
The cost to get something in the queue for a developer consists of:
- Identifying a customer need
- Describing it as a feature
- Estimating its cost of deliver (time, effort, etc)
- Prioritizing it for a release
- Placing it into the developer's queue.
Once it is in the developer's queue I do not know of any significant cost for the developer to manage the queue. So, limiting WIP for a developer seems unnecessary to me. However, if items are thrown into a developers queue that have not been through the above steps and the burden falls on the developer to define, prioritize, or other tasks then there is cost to the developer.
If the developer's queue is in constant flux there could be reason to limit WIP. If product can't make up their mind on direction and one day feature A is priority one and the next feature B, and the next a whole new and never heard of feature C and then back to A then there is cost for the developer. (Are process constraints the right way to address indecisiveness?) In this case a developer's queue could be limited to five items, or one iteration's worth of tasks, and once the iteration has started the queue cannot be altered. But this is process trying to mitigate a more serious problem of indecisiveness. Granted it should be mitigated for a short term help but it is not the solution. Maybe the limited WIP could help gather metrics of cost to show to managers how expensive it is to jump from one high priority task to another, back and forth.
But in a stable system limiting WIP at the developer doesn't seem advantageous to me yet. Maybe there are those that have thousands of items in the backlog. That would be a pain to manage.
Limiting WIP and the product level does seem reasonable to me. Image you have thought up enough features for three product releases for a desktop application that is released annually.
If all of those features are placed in a product queue and defined as follows:
- Identifying a customer need
- Describing it as a feature
- Estimating its cost of deliver (time, effort, etc)
- Prioritizing it for a release
Features that do not make it in the current release need not nor should they be well defined because things will change and when it is time to place it in release queue much of the work will have been wasted. That is why "Agile" methods recommend using a high level description, one sentence, something to capture the thought and cause the proper conversation to happen at a future date.
Therefore limiting WIP at the release queue seems very reasonable to me. The limit is this, only fully prepare an item for release if it is in the current release.
Software features are perishable goods. Image a software feature as an apple. If you worked at an apple pealing station and your bin of apples had enough in there to keep you busy all year, I wouldn't want any of your apples after two or three works of work!
Friday, July 30, 2010
Thursday, July 29, 2010
Work In Progress (WIP) and Little's Law
I recently made a small writeup concerning Work In Progress (WIP). I described that throughput is the results of the amount of work to be done divided by the time it takes to do the work. I threw the writeup out yesterday when I was pointed to Little's law, which is exactly what I was defining.
www.factoryphysics.com/Principle/LittlesLaw.htm
(Note that I have had experience with line based work since a child. I grew up on a dairy farm and we went from a simple stanchion barn to what is called a double herringbone with 4 stalls on each side. This was useful for two workers but when there was only one it was too much and we removed 1 stall from each side so that it was a double 3)
I am still reading about Little's law, but this much I have noticed:
The key to the Inventory = Throughput X Flow Time is consistency and lack of variability in units of measure.
Note that even using "cost" as the common unit it is still difficult to use in software development.
Cost is unknown or varies.
Time is unknown or varies.
Complexity is unknown or varies.
Congestion is unknown or varies.
Bottlenecks are unknown or vary.
All of the above issues have been concerns of every software process and software estimation technique. But note that even the best techniques still give estimations.
Predictability is becoming (or maybe already is) the main selling point for software process. This is because if I were going to attempt to use Little's Law for software development I would have to bring software development into the realm of manufacturing and to do this I need predictability. In other words, I can do things to the software process to make it seem to fit better and then jump to the conclusion that if it fits well enough then Little's law still must hold.
If I could get every developer on my team to divide any task into equal chunks of work then I could apply Little's law. If any task no matter the size or difficulty can be divided, redivided, and re-redivided until it is a small and consistent chunk then I can apply Little's law. If the cost of subdividing large features is to much then I can argue that there should be no large features. I could argue no large features really means that all features can be delivered incrementally and therefore there is no need for any large features.
All of that arguing takes you down, not only a very narrow path, but one that is not necessarily true.
For instance, releasing a product into a market that is well established requires large feature development. If I were going to break into the word processing market my first release of the software could only have plain text, no wrapping, etc., and then I would incrementally release wrapping, fonts, until finally the product has a feature set that is comparable to the competition.
The "total incremental" delivery approach implies that all features can be evolved. (I would like to see that proof.)
Now, can there be things learned from Little's law? Certainly. Maybe we could have a discussion about that and find ways to use it to improve software development.
For instance, if you find you are enhancing exist software and roll-out is done regularly and consistently then you may be in a situation where you should subdivide large features and get your development down to the smallest reasonable "chunk size" as possible.
I am doing further investigation based on this:
"Reducing WIP in a line without making any other changes will also reduce throughput."
Geoff
p.s.
(Additional Research)
www.shmula.com/2407/lean-and-theory-of-constraints-either-or
www.factoryphysics.com/Principle/LittlesLaw.htm
(Note that I have had experience with line based work since a child. I grew up on a dairy farm and we went from a simple stanchion barn to what is called a double herringbone with 4 stalls on each side. This was useful for two workers but when there was only one it was too much and we removed 1 stall from each side so that it was a double 3)
I am still reading about Little's law, but this much I have noticed:
The key to the Inventory = Throughput X Flow Time is consistency and lack of variability in units of measure.
Note that even using "cost" as the common unit it is still difficult to use in software development.
Cost is unknown or varies.
Time is unknown or varies.
Complexity is unknown or varies.
Congestion is unknown or varies.
Bottlenecks are unknown or vary.
All of the above issues have been concerns of every software process and software estimation technique. But note that even the best techniques still give estimations.
Predictability is becoming (or maybe already is) the main selling point for software process. This is because if I were going to attempt to use Little's Law for software development I would have to bring software development into the realm of manufacturing and to do this I need predictability. In other words, I can do things to the software process to make it seem to fit better and then jump to the conclusion that if it fits well enough then Little's law still must hold.
If I could get every developer on my team to divide any task into equal chunks of work then I could apply Little's law. If any task no matter the size or difficulty can be divided, redivided, and re-redivided until it is a small and consistent chunk then I can apply Little's law. If the cost of subdividing large features is to much then I can argue that there should be no large features. I could argue no large features really means that all features can be delivered incrementally and therefore there is no need for any large features.
All of that arguing takes you down, not only a very narrow path, but one that is not necessarily true.
For instance, releasing a product into a market that is well established requires large feature development. If I were going to break into the word processing market my first release of the software could only have plain text, no wrapping, etc., and then I would incrementally release wrapping, fonts, until finally the product has a feature set that is comparable to the competition.
The "total incremental" delivery approach implies that all features can be evolved. (I would like to see that proof.)
Now, can there be things learned from Little's law? Certainly. Maybe we could have a discussion about that and find ways to use it to improve software development.
For instance, if you find you are enhancing exist software and roll-out is done regularly and consistently then you may be in a situation where you should subdivide large features and get your development down to the smallest reasonable "chunk size" as possible.
I am doing further investigation based on this:
"Reducing WIP in a line without making any other changes will also reduce throughput."
Geoff
p.s.
(Additional Research)
www.shmula.com/2407/lean-and-theory-of-constraints-either-or
Thursday, July 22, 2010
Agile and "People before Process"
As I have watched this thing called "Agile" develop I have always been looking for things to creep in that may be in violation of foundational aspects of Agile process.
Okay, I do more than watch, I try to direct where I feel direction is needed. It might not be wanted but it is what I "feel" is needed. One thing that took me a while to learn is that my opinion and observation is just as good as anyone's. I tended to defer too much because I didn't understand that some people are just "word" factories continuously spewing forth informational pollution. Maybe I have become a polluter.
Agile has been described and re-described and defended over an over. This is a refinement type process. It is natural.
A recent description of an Agile team is "a team who's members have all of the skills necessary to complete the task." If the team doesn't have the exactly right members then it has failed this Agile qualification and therefore is not Agile and any failures cannot be attributed to Agile.
Well, I am going to make up a new software process right here and right now. It is called "Superior Software Development Process" or SSDP.
Superior Software Development Process requires omnipotent and omniscient members. SSDP guarantees success. If you have mere mortals on your team you cannot use SSDP.
People before process. I think that is important. I recognize that people are imperfect. Sometimes imperfect people luck up and do better than expected and sometimes they chose incorrectly due to lack of experience and really mess things up.
An Agile team is made up of imperfect people with limited skill sets who are willing to learn and with new information and experience continuously refine estimates and improve code quality, product quality, and user satisfaction.
Agile recognizes this is a journey, that this is a growth process, that the end cannot be seen but we can guess what is coming and accept what actually comes.
Okay, I do more than watch, I try to direct where I feel direction is needed. It might not be wanted but it is what I "feel" is needed. One thing that took me a while to learn is that my opinion and observation is just as good as anyone's. I tended to defer too much because I didn't understand that some people are just "word" factories continuously spewing forth informational pollution. Maybe I have become a polluter.
Agile has been described and re-described and defended over an over. This is a refinement type process. It is natural.
A recent description of an Agile team is "a team who's members have all of the skills necessary to complete the task." If the team doesn't have the exactly right members then it has failed this Agile qualification and therefore is not Agile and any failures cannot be attributed to Agile.
Well, I am going to make up a new software process right here and right now. It is called "Superior Software Development Process" or SSDP.
Superior Software Development Process requires omnipotent and omniscient members. SSDP guarantees success. If you have mere mortals on your team you cannot use SSDP.
People before process. I think that is important. I recognize that people are imperfect. Sometimes imperfect people luck up and do better than expected and sometimes they chose incorrectly due to lack of experience and really mess things up.
An Agile team is made up of imperfect people with limited skill sets who are willing to learn and with new information and experience continuously refine estimates and improve code quality, product quality, and user satisfaction.
Agile recognizes this is a journey, that this is a growth process, that the end cannot be seen but we can guess what is coming and accept what actually comes.
End of Iteration is not the "final exam"
The end of a development iteration should not be considered the final exam.
During a semester at college for a given class there were many assignments to be turned in, there were quizzes, mid-term exams, more assignments and quizzes, and the final exam.
When software development organizations adopt an iterative development process I have observed that the developers tend to over react to the iteration aspects of iterative development and forget about the full Product Plan.
New iterative teams seem to overreact to anything that might interfere with their "well-define" iteration. If you get to the end of your iteration (and I am thinking about one to two week iterations) and you got less done or you got more done it is okay. The end of an iteration should not be viewed as a "final exam" and should not carry that type of weight.
I believe that developers are afraid they will be judged on the results of the end of an iteration in the same manner they would be judged on the results of a product release. This should not be the case.
I believe that managers are afraid as well and some how feel they need to show that they can accurately predict software delivery after ten or fewer iterations of release schedule that is made up of fifty or more iterations.
I know of "agile" managers that refuse to add something to an iteration even if the developer got all of the other tasks done and there are two more days left. Is velocity really that precious? Is the manager's ability to sell up the idea that this manager can accurately predict and estimate software to such a fine degree really what is important?
There is an entire Project/Product plan. Features and enhancement are gathered. These are divided into releases that could cover several versions of the product over several years. Releases are divided into iterations. Iterations are quizzes.
Maybe there should be some type of "mid-term" exam where the current set of functionality is deployed to a test environment or a test installer just to see where things are at and that we are not forgetting something (like securing licenses to redistribute 3rd part components).
Lighten up on the end of iteration stress that is artificial and not needed.
During a semester at college for a given class there were many assignments to be turned in, there were quizzes, mid-term exams, more assignments and quizzes, and the final exam.
When software development organizations adopt an iterative development process I have observed that the developers tend to over react to the iteration aspects of iterative development and forget about the full Product Plan.
New iterative teams seem to overreact to anything that might interfere with their "well-define" iteration. If you get to the end of your iteration (and I am thinking about one to two week iterations) and you got less done or you got more done it is okay. The end of an iteration should not be viewed as a "final exam" and should not carry that type of weight.
I believe that developers are afraid they will be judged on the results of the end of an iteration in the same manner they would be judged on the results of a product release. This should not be the case.
I believe that managers are afraid as well and some how feel they need to show that they can accurately predict software delivery after ten or fewer iterations of release schedule that is made up of fifty or more iterations.
I know of "agile" managers that refuse to add something to an iteration even if the developer got all of the other tasks done and there are two more days left. Is velocity really that precious? Is the manager's ability to sell up the idea that this manager can accurately predict and estimate software to such a fine degree really what is important?
There is an entire Project/Product plan. Features and enhancement are gathered. These are divided into releases that could cover several versions of the product over several years. Releases are divided into iterations. Iterations are quizzes.
Maybe there should be some type of "mid-term" exam where the current set of functionality is deployed to a test environment or a test installer just to see where things are at and that we are not forgetting something (like securing licenses to redistribute 3rd part components).
Lighten up on the end of iteration stress that is artificial and not needed.
Resources versus Teams
Scenario
Suppose you find yourself at a software company that has 80 or more developers. The developers are assigned to teams which in turn are aligned to specific products, features, and business needs. Over the past ten years so so the company has hired and fired and people have moved around and found a team or product area that they like.
The company is at a stage where it wants to deliver several initiatives that it feels will excite customers and frustrate competition. Some of the new products are defined to pull together data and functionality across teams because it is obvious to unify product functionality and leverage established value.
The release schedule and product descriptions are presented to development. Teams will be required to integrate their systems. Development raises concerns because of the cost of integrating systems, team communication, scheduling, dependencies, and work that must be done on the existing systems in order to implement specific enhancements.
Response
Product Management is alarmed by the concerns of Development.
Why is it that the software doesn't readily integrate? "We" have been working together for ten years. Are the Architects doing there job?
Why is communication going to be costly? Where is the professionalism that is needed at this time? Why are we silo'ed?
Possible Reaction by Product Personnel
Product Management looks at the developers and recognizes them as highly skilled employees that are concerned with the big picture and with company wide goals.
We must break down these walls and tear down these silos. "What if we move developers around as needed, you know, if product 'X' needs integration with product 'Y' we move the people we need to get the job done and align the reporting chain accordingly." That is being agile.
My Remarks
Be very careful when viewing developers, which are people, as resources. Be very careful when discounting the usefulness of teams, which are groups of people. Be very careful thinking that all you have to do is get the right skill sets/resources together and that will almost guarantee the likelihood of success.
If you have development which seems silo'ed did you ever stop to think that maybe that is just a negative adjective "silo'ed" and what you really have is development divided along products that run efficiently and almost autonomously?
I argue that every team has some aspects of self organization. Through the hiring, the firing, and the team transfers, teams settle into a state that is acceptable both to the team members and well as to management. Even if your team is ran by a dictator you can still quit so anyone that remains on the team does so by choice.
I argue that any outward examination of a development organization may describe the organization as silo'ed. A well defined organization will always have well defined attributes that may be described as walls. Any skilled debater knows to use adjectives that are commonly viewed as a negativity to advocate their agenda. Possibly this silo'ed and walled development organization is really a self organized team aligned specifically to product needs and technological needs.
Possible Solution
Listen to development's concerns and then give them a week to propose solutions. If development's concerns are not about implementation and delivery but are about the products and features themselves then management may need to take the time to explain why these are the "right" products and features. This can be done with information from real customers. If you are thinking, "Developers don't need to know why we want these products and features they just need to do the job they were hired to do which is write software and implement the things we demand", then you should reconsider what a developer really is. A developer is more than a coder. Developers use software all day, every day, and have valuable experience in recognizing good software.
Developers understand that when deciding a product schedule that costs are to be considered and some ideas will not make it to the table because of some unacceptable attribute. They know that some ideas for a product can be described easily and with few words but just because everyone can understand an idea because of common experience which gives way for simple descriptions doesn't mean the software will be simple, short, or easy to develop.
Listen to the concerns of the developers and then give them a week to come up with solutions. Developers are problem solvers. Let them solve the problem. If development comes back and says the product cannot be developed then you have to ask why? It can't be developed because it will take longer than acceptable? It can't be developed because the existing list of things to do are higher priority or have been overlooked to often and these things have to be done before anyone can even guess what additional functionality might cost in time and effort. It can't be done because the technologies you want to integrate will not integrate because of data mapping issues.
There are many reasons why things cannot be done now. An interesting question is, "If it can't be done now what does it take to get things so that we can do it in the future and when will that be?"
Listen and ask for solutions and give time to come up with solutions.
Pet Peeve
I really cannot stand it when product people think up "cool" features and products and sell them up before selling them down.
As a developer I have said, "Wouldn't it be a sweet job for me to dream up really cool stuff that someone else has to develop." Work together.
Don't sell your boss on something and then find development saying it can't be done. Do you really want to go to your boss now and say that your team can't deliver what you sold him? Do you really want to press your developers just to save your face with the boss? Don't get yourself into that situation.
Maybe the silo and the wall is with the product people and the development teams. Maybe "Product" has silo'ed all of the activities of product and feature definition. Ah, I am glad you noticed that I used the term silo'ed with all of its negative connotations. Maybe I am becoming a skilled debater.
Suppose you find yourself at a software company that has 80 or more developers. The developers are assigned to teams which in turn are aligned to specific products, features, and business needs. Over the past ten years so so the company has hired and fired and people have moved around and found a team or product area that they like.
The company is at a stage where it wants to deliver several initiatives that it feels will excite customers and frustrate competition. Some of the new products are defined to pull together data and functionality across teams because it is obvious to unify product functionality and leverage established value.
The release schedule and product descriptions are presented to development. Teams will be required to integrate their systems. Development raises concerns because of the cost of integrating systems, team communication, scheduling, dependencies, and work that must be done on the existing systems in order to implement specific enhancements.
Response
Product Management is alarmed by the concerns of Development.
Why is it that the software doesn't readily integrate? "We" have been working together for ten years. Are the Architects doing there job?
Why is communication going to be costly? Where is the professionalism that is needed at this time? Why are we silo'ed?
Possible Reaction by Product Personnel
Product Management looks at the developers and recognizes them as highly skilled employees that are concerned with the big picture and with company wide goals.
We must break down these walls and tear down these silos. "What if we move developers around as needed, you know, if product 'X' needs integration with product 'Y' we move the people we need to get the job done and align the reporting chain accordingly." That is being agile.
My Remarks
Be very careful when viewing developers, which are people, as resources. Be very careful when discounting the usefulness of teams, which are groups of people. Be very careful thinking that all you have to do is get the right skill sets/resources together and that will almost guarantee the likelihood of success.
If you have development which seems silo'ed did you ever stop to think that maybe that is just a negative adjective "silo'ed" and what you really have is development divided along products that run efficiently and almost autonomously?
I argue that every team has some aspects of self organization. Through the hiring, the firing, and the team transfers, teams settle into a state that is acceptable both to the team members and well as to management. Even if your team is ran by a dictator you can still quit so anyone that remains on the team does so by choice.
I argue that any outward examination of a development organization may describe the organization as silo'ed. A well defined organization will always have well defined attributes that may be described as walls. Any skilled debater knows to use adjectives that are commonly viewed as a negativity to advocate their agenda. Possibly this silo'ed and walled development organization is really a self organized team aligned specifically to product needs and technological needs.
Possible Solution
Listen to development's concerns and then give them a week to propose solutions. If development's concerns are not about implementation and delivery but are about the products and features themselves then management may need to take the time to explain why these are the "right" products and features. This can be done with information from real customers. If you are thinking, "Developers don't need to know why we want these products and features they just need to do the job they were hired to do which is write software and implement the things we demand", then you should reconsider what a developer really is. A developer is more than a coder. Developers use software all day, every day, and have valuable experience in recognizing good software.
Developers understand that when deciding a product schedule that costs are to be considered and some ideas will not make it to the table because of some unacceptable attribute. They know that some ideas for a product can be described easily and with few words but just because everyone can understand an idea because of common experience which gives way for simple descriptions doesn't mean the software will be simple, short, or easy to develop.
Listen to the concerns of the developers and then give them a week to come up with solutions. Developers are problem solvers. Let them solve the problem. If development comes back and says the product cannot be developed then you have to ask why? It can't be developed because it will take longer than acceptable? It can't be developed because the existing list of things to do are higher priority or have been overlooked to often and these things have to be done before anyone can even guess what additional functionality might cost in time and effort. It can't be done because the technologies you want to integrate will not integrate because of data mapping issues.
There are many reasons why things cannot be done now. An interesting question is, "If it can't be done now what does it take to get things so that we can do it in the future and when will that be?"
Listen and ask for solutions and give time to come up with solutions.
Pet Peeve
I really cannot stand it when product people think up "cool" features and products and sell them up before selling them down.
As a developer I have said, "Wouldn't it be a sweet job for me to dream up really cool stuff that someone else has to develop." Work together.
Don't sell your boss on something and then find development saying it can't be done. Do you really want to go to your boss now and say that your team can't deliver what you sold him? Do you really want to press your developers just to save your face with the boss? Don't get yourself into that situation.
Maybe the silo and the wall is with the product people and the development teams. Maybe "Product" has silo'ed all of the activities of product and feature definition. Ah, I am glad you noticed that I used the term silo'ed with all of its negative connotations. Maybe I am becoming a skilled debater.
Subscribe to:
Posts (Atom)