I will work from these definitions.
Software is a general term for the various kinds of programs used to operate computers and related devices.
A Program is a specific set of ordered operations for a computer to perform.
A Computer is a device that accepts information and operation instructions.
So, to rephrase the initial question, What is necessary to develop specific sets of ordered instructions for a computer to perform?
You will need:
- A computer
- A specification of the instructions the computer accepts
- Something to generate sequences of instructions
Something to generate sequences of instructions! Another computer could do this. A person could do this if the person has knowledge of the computer's instruction set.
A Programmer is a person that specifies a sequence of instructions for a computer.
I will not go into the roles of low level languages, high level languages, compilers, interpreters, and other methods of specifying computer instructions.
So, that is all that is necessary to develop computer programs.
What? You feel that there is more necessary? I don't think so.
Oh, you want to develop a specific program to solve a specific problem. Well then, it is necessary for you to describe the problem in terms such that the programmer can order the correct sequence of instructions for the computer to solve the problem.
Does the specification of the problem have to be done in a certain format? No. The only thing necessary is that you communicate to the programmer accurately the problem to be solved.
So, if you want a specific problem solved then you must define the requirements.
Now we are up to:
- A computer
- A specification of the instructions the computer accepts
- Something to generate sequences of instructions
- Specification of the problem requirements
Wait, does the problem specification include a description or specification of the solution? No, it does not. The expected results of a software program may be termed the solution set. For instance if the specification for the program says, "Enter two numbers and compute the multiplication of those two number and display them." If you enter the numbers 6 and 2 and the results where "To be or not to be? That is the question." clearly that result is not in the expected set of solutions for multiplication.
The specification of the solution set or correct set of results for a program are necessary if you only accept specific results.
Now we are up to:
- A computer
- A specification of the instructions the computer accepts
- Something to generate sequences of instructions
- Specification of the problem requirements
- Specification of acceptable results
Some might say the last to specifications go together into one specification. It doesn't matter to me. The point is, if you want specific problems to be solved and receive results in a specific set of possible results then it is necessary for those to be specified and communicated to the programmer.
So, that's all you need.
But you are thinking there is so much more to software development. There are books and books on how to write good solid code. There are books and books on how to organize teams. There are books and books on how to layout the User Interface.
Ultimately it all depends on what you want and what you value and the things you are willing to do to get the things you want.
Here are some related questions to the topic:
- What is necessary to develop software quickly?
- What is necessary to develop software with few defects?
- What is necessary to develop software that runs on many different computers?
- What is necessary to develop software inexpensively?
- What is necessary to stop developers from quitting?
- What is necessary to protect intellectual property?
- What is necessary to attract talented programmers?
- What is necessary to get incremental results from the development effort?
- What is necessary to build software for life critical systems?
- What is necessary to make code that is reusable?
- What is necessary to make code backwards compatible?
- What is necessary to develop software for moving/shifting requirements?
The list goes on and on and on.
Geoff
Friday, February 29, 2008
Thursday, February 28, 2008
Lessons learned from the farm.
My Dad has often recounted to me the story of a neighboring farmer. The neighbor ran a dairy where they milked Jersey cows. It was a very profitable operation. Not only was it ran well, it was clean and in good order, meaning that the barns were in good repair and that the fence rows were not grown up with briers, weeds, or trees.
The neighbor sent his son to a University were the son majored in agriculture. Upon his graduation and return to the farm the son started applying the lessons he learned at the University. The son took the savings from the farm's years of profit and increased the size of the operation. One of the changes was the adding of a Harvestore "big blue" silo. Trust me, these silos are expensive. The son ran the operation and soon was broke and had to sell out.
Now the son is working at the state government level in the Department of Agriculture.
My Dad is amazed that someone who took control of a "gem" of a dairy operation and ran it into the ground is qualified to make agriculture decisions at the state level.
In my opinion the man is not qualified.
When I remember this story it causes me to think about software development and a couple of similarities I have seen.
The first is that I have seen directors and executives run companies into the ground and then in mass those "people" go to another company and do it all over again. This scenario is not exclusive to dairy farms and software development. I know that using such derisive terms as "ran into the ground" poison the well and since I do not know all of the reasons for the company failures and thus this is an invalid argument based on an appeal to consequences of belief. Nevertheless I have seen people advance in the same business sector even though previously they were involved in significant failures in that same sector.
The second thing that the dairy farm story reminds me is that "just because it was taught at the University doesn't mean it will work!". I loved my education at Brigham Young University. The professors there taught me well. However, I knew the dairy farm story before I went to the University and I knew that I would have to carefully apply the lessons from school to my career.
So, some advice for my friends. When someone else has an idea on how you should run your business figure out if that someone makes a living doing what you do. The University Professor teaching agriculture makes his money through the University and not through farming. The Professor's teachings may be right and applicable and they may not be applicable. The onus or burden of proof falls upon you.
One might conclude that my Father's farm is not progressive because he doesn't follow the latest suggestions from the Universities, feed salesmen, or machinery salesmen. This is not so. My Dad has often told me of how his father had them using a mule to plow the fields. My Dad and his younger Brother decided that if they were going to run the farm and make any money doing it that they would need a tractor. So the two brothers bought a tractor and some implements to use with that tractor.
My Father would go to different parts of the U.S.A. and see what farmers where doing there. I remember when we had traveled through the Mid-West and we had seen the hay balers that make a large round bale. My Dad ordered one and our farm was the first in the area to use the new technology by several years.
My Father also rejected many ideas that he saw adopted. One of which was to place all of the cattle into a confined area to limit the cattle's movement and to put in large silos and conveyor systems. Our neighbors did so. To pay for the silos and such they added more cows to the heard. Even though our neighbors had an operation valued in the millions of dollars my Dad's farm made more profit, was less stressful on the cattle because the cows stilled roamed in their pasture, costs less in operations because the cattle are healthier, etc.
My Father rejected the recommendation of the Agriculture representatives to give the cows a shot of hormones, known as BST, to the cattle. The cost of the hormones was not the reason he rejected this idea. The idea of giving the cows a shot, and trust me the needle to give a cow a shot is large and it hurts the cow, and making the cows nervous and even mean was not worth it. Also, it was not something he wanted in his milk and so he figured you wouldn't want it in yours (even though they claimed that the hormone can not end up in the milk). The shots, the confinement, and other issues of a large "modern" dairy reduces the life of a cow from over 10 years to less than 7 years. The costs of raising or buying replacement cattle is tremendous.
My Father values the living conditions of his cattle over making a profit. My Father has learned that having a bigger business does not mean you have a more profitable business. My Father has learned that salesmen are motivated to sell their product and can spin quite the story of reasons why you should use their product. My Father does not define success by short term profits alone.
For those that have stuck with me on this post you may be asking what does this have to do with software development. To me it has plenty and if it hasn't been clear then please post some comments and I will try to help you understand.
Remember, you are probably smarter than you think you are and those you think are really smart may not be as smart as you think! Think for yourself. Know what is important to you and if you don't know what is important you will eventually learn what is important.
When someone is trying to sell you something that will make you money here's a little test for them:
If someone says that by adopting the new Lissom Software Development Methodology you will increase your productivity by 20%, then ask them to put it into writing and if it doesn't deliver they will give you your money back!
When they refuse to offer you a money back guarantee ask them to stop with the hype and spin and ask them to work with you so that you can see if their methodology addresses real problems that you are experiencing.
You know the problems you face even though you might not recognize the root cause of the problems and you might not know all of the possible ways to address the problem. That is where getting help and ideas from others internally or externally can be useful.
Just some thoughts.
The neighbor sent his son to a University were the son majored in agriculture. Upon his graduation and return to the farm the son started applying the lessons he learned at the University. The son took the savings from the farm's years of profit and increased the size of the operation. One of the changes was the adding of a Harvestore "big blue" silo. Trust me, these silos are expensive. The son ran the operation and soon was broke and had to sell out.
Now the son is working at the state government level in the Department of Agriculture.
My Dad is amazed that someone who took control of a "gem" of a dairy operation and ran it into the ground is qualified to make agriculture decisions at the state level.
In my opinion the man is not qualified.
When I remember this story it causes me to think about software development and a couple of similarities I have seen.
The first is that I have seen directors and executives run companies into the ground and then in mass those "people" go to another company and do it all over again. This scenario is not exclusive to dairy farms and software development. I know that using such derisive terms as "ran into the ground" poison the well and since I do not know all of the reasons for the company failures and thus this is an invalid argument based on an appeal to consequences of belief. Nevertheless I have seen people advance in the same business sector even though previously they were involved in significant failures in that same sector.
The second thing that the dairy farm story reminds me is that "just because it was taught at the University doesn't mean it will work!". I loved my education at Brigham Young University. The professors there taught me well. However, I knew the dairy farm story before I went to the University and I knew that I would have to carefully apply the lessons from school to my career.
So, some advice for my friends. When someone else has an idea on how you should run your business figure out if that someone makes a living doing what you do. The University Professor teaching agriculture makes his money through the University and not through farming. The Professor's teachings may be right and applicable and they may not be applicable. The onus or burden of proof falls upon you.
One might conclude that my Father's farm is not progressive because he doesn't follow the latest suggestions from the Universities, feed salesmen, or machinery salesmen. This is not so. My Dad has often told me of how his father had them using a mule to plow the fields. My Dad and his younger Brother decided that if they were going to run the farm and make any money doing it that they would need a tractor. So the two brothers bought a tractor and some implements to use with that tractor.
My Father would go to different parts of the U.S.A. and see what farmers where doing there. I remember when we had traveled through the Mid-West and we had seen the hay balers that make a large round bale. My Dad ordered one and our farm was the first in the area to use the new technology by several years.
My Father also rejected many ideas that he saw adopted. One of which was to place all of the cattle into a confined area to limit the cattle's movement and to put in large silos and conveyor systems. Our neighbors did so. To pay for the silos and such they added more cows to the heard. Even though our neighbors had an operation valued in the millions of dollars my Dad's farm made more profit, was less stressful on the cattle because the cows stilled roamed in their pasture, costs less in operations because the cattle are healthier, etc.
My Father rejected the recommendation of the Agriculture representatives to give the cows a shot of hormones, known as BST, to the cattle. The cost of the hormones was not the reason he rejected this idea. The idea of giving the cows a shot, and trust me the needle to give a cow a shot is large and it hurts the cow, and making the cows nervous and even mean was not worth it. Also, it was not something he wanted in his milk and so he figured you wouldn't want it in yours (even though they claimed that the hormone can not end up in the milk). The shots, the confinement, and other issues of a large "modern" dairy reduces the life of a cow from over 10 years to less than 7 years. The costs of raising or buying replacement cattle is tremendous.
My Father values the living conditions of his cattle over making a profit. My Father has learned that having a bigger business does not mean you have a more profitable business. My Father has learned that salesmen are motivated to sell their product and can spin quite the story of reasons why you should use their product. My Father does not define success by short term profits alone.
For those that have stuck with me on this post you may be asking what does this have to do with software development. To me it has plenty and if it hasn't been clear then please post some comments and I will try to help you understand.
Remember, you are probably smarter than you think you are and those you think are really smart may not be as smart as you think! Think for yourself. Know what is important to you and if you don't know what is important you will eventually learn what is important.
When someone is trying to sell you something that will make you money here's a little test for them:
If someone says that by adopting the new Lissom Software Development Methodology you will increase your productivity by 20%, then ask them to put it into writing and if it doesn't deliver they will give you your money back!
When they refuse to offer you a money back guarantee ask them to stop with the hype and spin and ask them to work with you so that you can see if their methodology addresses real problems that you are experiencing.
You know the problems you face even though you might not recognize the root cause of the problems and you might not know all of the possible ways to address the problem. That is where getting help and ideas from others internally or externally can be useful.
Just some thoughts.
Sunday, February 17, 2008
Poor Requirements, Poor Coding, Poor Design, Poor Policies, Poor Management, Poor Leadership
The software development industry seems to be completely in love with excuses! (Sadly enough, it seems that many industries are infected with this poor behavior. Pun intended.)
In software development, when a problem is identified it doesn't take but an instance to pass before someone has an explanation. Maybe drawing rapid conclusions is part of the software development ecosystem. Developing software requires quick decision making. Maybe "quick thinking" has become our primary method of problem solving.
I have seen many of the different problems encountered from software development. Buggy software or quality problems, performance issues, missing features, extra features, months of overtime, changing requirements and moving targets, threats used to squeeze out a little more work, heroics, power plays, re-organizations, firings, and more.
If you work for a company who's products are web based have you ever experienced something that brought down "the site"? At least that is what it is referred to in the Utah area, that is "bringing down the site". Has this happened where you work?
I have seen it happen a few times. The most drastic reactions from bringing down the site have resulting in the CEO calling the developer for an explanation AND the CTO coming down to "see" what the problem really is. It was termed the "blame game". Someone has to be blamed. Maybe the Executives where not looking for someone to blame but instead were offering suggestions from a higher view hoping their perspective could help. Regardless of their unknown intentions the interpretation was that someone had to be blamed.
The blaming resulted in excuses. Excuses such as, "My change brought the site down because of the ridiculous policy that I can't have access to the live databases. I did not know that the tables on live did not have the exact same schema as those on our staging machines", or, "My change brought down live because the design of the system is so complex that there is no way I can manually test all of the possible states the system may enter", or, "The system that you forced us to build upon is very complex and the company has never trained us on its proper use."
So, back to the title of this post.
"Poor Requirements, Poor Coding, Poor Design, Poor Policies, Poor Management, Poor Leadership"
Upon release of a product if it doesn't do what is wanted then the excuse is "poor requirements". It may be true that the requirements were poorly defined. But waiting to the release of the product to discover this is ridiculous. It is ridiculous on the part of management, on part of the customer, and on part of the developer. Historically the suggested solution to this problem has been creating a more formal and controlled requirements process. A more formal process is something management can dictate but apart from large amounts of ceremony have failed to deliver.
I really don't care how long it takes a company to define the requirements for a software product. There are many that love to play in the undefined space of "Big Upfront Requirements". Well, they can play there all they want. It is a situational trap.
I do care that the company was foolish enough to wait until the product is finished to start finding fault with it and to start looking for those to blame. Those responsible for the delivery of a software product should be using that product continuously during it's development. There is no excuse for not doing so. If those concerned say, "The software is too new, too buggy, and won't run long enough to really evaluate it. So we will wait until they make their alpha candidate." Hog wash. First there are formal management techniques of functional decomposition which allows for meaningful incremental development and release of working features. If management wants to put something formal in place, then put in incremental development.
If you think that incremental development is ad hoc, or some type of cowboy approach then you do not understand it. Incremental developer imposes a level of work that waterfall development does it. It requires features to be organized, group by dependency and priority, estimated, and the scheduled by placing them into release queues. What I just described is a lot of work. A lot of work before the first line of code is developed. Notice I did not say the feature was designed at the code level. That is something different that I will talk about at a future date.
Do you see how these complaints of poor this and poor that are just excuses?
I have lightly covered poor requirements. I will touch on poor management. I know that many believe that a manager of developers doesn't have to be a developer. I will agree with this if the manager has the ability to know where the development line is drawn.
I had a manager that had did some programming. Delphi type stuff, some HTML, a few databases designed with a nice GUI DB designer. Those kinds of things. As sophisticated as that is compared to the masses, it is light weight stuff compared to systems level development using multiple processes communicated using shared memory techniques or wiring up a model view controller in such a way as to avoid race conditions. However this manager (director level) assumed that he really understood programming and he would come down and stick his big honking management nose in the developers code. He required developers of 20+ years experience to explain all of their code changes and designs to him for approval. When we faced technical issues he would make design level decisions that were so idiotic that we couldn't believe it. There was a problem with some web servers that were not responding to sent messages. He said to the team, "If the server fails to respond, send the message again. Double pump the message. Triple pump the message. We must be sure the message gets to the server." How utterly ignorant of server side development! Sending more and more messages to a server that is not responding is not the first solution to the problem. Before I go completely insane from remembering these experiences I better get back on topic. We had poor management.
Poor management was our excuse for our problems. Because we had poor management we felt that we couldn't fix the real problems. Those were excuses and we did not allow excuses to stop us from working to the best of our ability. We did not follow his advice of double pumping the messages and we did not tell him how we fixed the problems. We chose to ignore him and do our job. It is hard to ignore a director when your pay raise goes through his office. But we did anyway. I left the team as did others. Soon there was a new director.
Ultimately I realize that I will be plagued with excuses and will be guilty of giving excuses. If you do not have the ability to make a change then you will probably make an excuse.
Surely there are many poor aspects of any software development. They will stay poor until someone realizes how to stop complaining and to start thinking and then doing something about. Just like the example I give of missed requirements. It is not reasonable to have such problems with the knowledge of incremental feature development.
As for the problem with the poor manager/director, the solution is one of communication and trust. That is a large topic. But to make it short either the manager could have learned to communicate his concern for quality at the same time showing that he trusted the developers to deliver, OR, the VP could have communicated with the developers and could have realized that the manager should be replaced, OR some other solution.
If your development process is suffering from something that is poor, then drop a line to us on this blog or on the users group, "http://tech.groups.yahoo.com/group/digerati-illuminatus/" and we will be glad to offer suggestions. For FREE! But be prepared to give open and honest details about the entire problem and the organization in which the problem lives.
In software development, when a problem is identified it doesn't take but an instance to pass before someone has an explanation. Maybe drawing rapid conclusions is part of the software development ecosystem. Developing software requires quick decision making. Maybe "quick thinking" has become our primary method of problem solving.
I have seen many of the different problems encountered from software development. Buggy software or quality problems, performance issues, missing features, extra features, months of overtime, changing requirements and moving targets, threats used to squeeze out a little more work, heroics, power plays, re-organizations, firings, and more.
If you work for a company who's products are web based have you ever experienced something that brought down "the site"? At least that is what it is referred to in the Utah area, that is "bringing down the site". Has this happened where you work?
I have seen it happen a few times. The most drastic reactions from bringing down the site have resulting in the CEO calling the developer for an explanation AND the CTO coming down to "see" what the problem really is. It was termed the "blame game". Someone has to be blamed. Maybe the Executives where not looking for someone to blame but instead were offering suggestions from a higher view hoping their perspective could help. Regardless of their unknown intentions the interpretation was that someone had to be blamed.
The blaming resulted in excuses. Excuses such as, "My change brought the site down because of the ridiculous policy that I can't have access to the live databases. I did not know that the tables on live did not have the exact same schema as those on our staging machines", or, "My change brought down live because the design of the system is so complex that there is no way I can manually test all of the possible states the system may enter", or, "The system that you forced us to build upon is very complex and the company has never trained us on its proper use."
So, back to the title of this post.
"Poor Requirements, Poor Coding, Poor Design, Poor Policies, Poor Management, Poor Leadership"
Upon release of a product if it doesn't do what is wanted then the excuse is "poor requirements". It may be true that the requirements were poorly defined. But waiting to the release of the product to discover this is ridiculous. It is ridiculous on the part of management, on part of the customer, and on part of the developer. Historically the suggested solution to this problem has been creating a more formal and controlled requirements process. A more formal process is something management can dictate but apart from large amounts of ceremony have failed to deliver.
I really don't care how long it takes a company to define the requirements for a software product. There are many that love to play in the undefined space of "Big Upfront Requirements". Well, they can play there all they want. It is a situational trap.
I do care that the company was foolish enough to wait until the product is finished to start finding fault with it and to start looking for those to blame. Those responsible for the delivery of a software product should be using that product continuously during it's development. There is no excuse for not doing so. If those concerned say, "The software is too new, too buggy, and won't run long enough to really evaluate it. So we will wait until they make their alpha candidate." Hog wash. First there are formal management techniques of functional decomposition which allows for meaningful incremental development and release of working features. If management wants to put something formal in place, then put in incremental development.
If you think that incremental development is ad hoc, or some type of cowboy approach then you do not understand it. Incremental developer imposes a level of work that waterfall development does it. It requires features to be organized, group by dependency and priority, estimated, and the scheduled by placing them into release queues. What I just described is a lot of work. A lot of work before the first line of code is developed. Notice I did not say the feature was designed at the code level. That is something different that I will talk about at a future date.
Do you see how these complaints of poor this and poor that are just excuses?
I have lightly covered poor requirements. I will touch on poor management. I know that many believe that a manager of developers doesn't have to be a developer. I will agree with this if the manager has the ability to know where the development line is drawn.
I had a manager that had did some programming. Delphi type stuff, some HTML, a few databases designed with a nice GUI DB designer. Those kinds of things. As sophisticated as that is compared to the masses, it is light weight stuff compared to systems level development using multiple processes communicated using shared memory techniques or wiring up a model view controller in such a way as to avoid race conditions. However this manager (director level) assumed that he really understood programming and he would come down and stick his big honking management nose in the developers code. He required developers of 20+ years experience to explain all of their code changes and designs to him for approval. When we faced technical issues he would make design level decisions that were so idiotic that we couldn't believe it. There was a problem with some web servers that were not responding to sent messages. He said to the team, "If the server fails to respond, send the message again. Double pump the message. Triple pump the message. We must be sure the message gets to the server." How utterly ignorant of server side development! Sending more and more messages to a server that is not responding is not the first solution to the problem. Before I go completely insane from remembering these experiences I better get back on topic. We had poor management.
Poor management was our excuse for our problems. Because we had poor management we felt that we couldn't fix the real problems. Those were excuses and we did not allow excuses to stop us from working to the best of our ability. We did not follow his advice of double pumping the messages and we did not tell him how we fixed the problems. We chose to ignore him and do our job. It is hard to ignore a director when your pay raise goes through his office. But we did anyway. I left the team as did others. Soon there was a new director.
Ultimately I realize that I will be plagued with excuses and will be guilty of giving excuses. If you do not have the ability to make a change then you will probably make an excuse.
Surely there are many poor aspects of any software development. They will stay poor until someone realizes how to stop complaining and to start thinking and then doing something about. Just like the example I give of missed requirements. It is not reasonable to have such problems with the knowledge of incremental feature development.
As for the problem with the poor manager/director, the solution is one of communication and trust. That is a large topic. But to make it short either the manager could have learned to communicate his concern for quality at the same time showing that he trusted the developers to deliver, OR, the VP could have communicated with the developers and could have realized that the manager should be replaced, OR some other solution.
If your development process is suffering from something that is poor, then drop a line to us on this blog or on the users group, "http://tech.groups.yahoo.com/group/digerati-illuminatus/" and we will be glad to offer suggestions. For FREE! But be prepared to give open and honest details about the entire problem and the organization in which the problem lives.
Tuesday, February 05, 2008
Agile Fiction and Myth
"Perhaps the sentiments contained in the following pages, are not YET sufficiently fashionable to procure them general favour; a long habit of not thinking a thing WRONG, gives it a superficial appearance of being RIGHT, and raises at first a formidable outcry in defense of custom. But the tumult soon subsides. Time makes more converts than reason." Common Sense, by Thomas Paine - 1776
The topic of this posting will be "The Agile Method and Other Fairy Tales", by David Longstreet.
http://www.softwaremetrics.com/Agfa/Agile%20Paper.pdf
Mr. Longstreet gives an impressive background of travel and study of software organizations in many varied fields and marketplaces. I declares that he has "been dedicated to the idea of improving software productivity and quality...consulting and studying software organizations".
I have twenty three years experience in software development. I have always been involved with the improvement of software quality and productivity. My level of involvement is "applied", that is, I am a Computer Scientist and Software Engineer. I apply ideas for improvement and I stay around to live with the results. I have a vested interested in improving software development because those improvements directly affect me. I have developed software on the family dairy farm to manage accounts payable and production. I have delivered software for the Department of Energy to visualize magnetic fields, and to render 3D data visualizations from scanning/tunneling microscopes. I have written coding standards and guides which were adopted by the entire Computer Science department at the National Laboratory where I worked. I have delivered software for EMail applications, real-time stock market analysis, record managers for high performance indexing engines, eCommerce systems, new specialized GUI controls, charting packages, and many many other high quality and working systems. I can produce references if someone wants to question the veracity of my statements.
I have not limited myself to just studying the software industry. I am an award winning Dairy farmer and have been recognized for my techniques in raising dairy calves. I have traveled and performed voluntary service. I have studied Gothic Architecture and Renaissance Art, and I have traveled in Western Europe to see the actual works. I study languages and culture. I have done in-depth research on the Biblical Prophet Abraham and I currently study the Qur'an. I collect HO scale electric trains and I change the oil in my automobiles. I played the Cornet and Saxophone and I have proven myself to be a fairly good artist. Like everyone I have met, I am not a one-dimensional person.
I have developed software in 68000 and VAX assembly, Pascal, FORTAN 77, Ada, APL, Object C, Object Pascal, C, C++, Java, HyperTalk, C#, various scripting languages, and SQL stored procedures. I have managed teams of developers. I do not limit myself to my department either. Everyone knows that I will speak up in other departments and I am not afraid to go to the CEO or the Board. As a matter of fact, I have been demanding on many organizations and I have had no fear of any position.
I state all of this because Mr. Longstreet does so at the beginning of his paper and then later in the paper describes Agile users as one dimensional. Even with all of my experience there are many things to learn.
Like Mr. Longstreet, I was very skeptical of eXtreme Programming (XP) when I first heard of it. I started writing a paper, XP eXposed. As I studied XP and then applied parts of it I started to see what Kent Beck was describing. Soon I abandoned the paper and started applying parts of XP and I found value in XP.
I will now take some quotes from Longstreet's paper and make some comments:
"I have come to the conclusion that software developers cause most of their own problems. The root cause of most of the problems facing software development is actually caused by software developers themselves. They are creating their own complexity and chaos."
This argument is flawed and is known as an appeal to consequences of a belief.
"Agile methods want continuation and formal acceptance of the status quo... Up to this point in time software development has been a Wild West endeavor... IT has been sloppy. There is nothing new with Agile, because it only tries to formalize sloppiness."
This argument is flawed and is known as an appeal to ridicule, spite, and ultimately the argument turns to an appeal of tradition.
"I am bringing a level of professionalism and rigor to the software industry, and I hope you join me."
This is a setup for the well known fallacy known as an "appeal to authority".
"An Agile proponent will argue there is limited value in requirements specifications because the requirements are ever changing."
There may be someone that Mr. Longstreet views as an Agile proponent that has argued this point. That does not make it so. Use Cases and User Stories are the two approaches I am most familiar with and both are taught and used by Agile proponents. I personally believe that some of the confusion is with the roles of XP and the idea of Agile.
"'I think it's fair to say that customer practices are not addressed in Agile methods.' It is clear that understanding what the customer wants or helping the customer figure out what they want is not really part of Agile, and in turn not part of software development."
Mr. Longstreet is quoting someone from an online users group. Such methods of fact finding are laughable. This statement is a hasty generalization, and hearsay.
"It is clear that understanding what the customer wants or helping the customer figure out what they want is not really part of Agile, and in turn not part of software development."
I believe this is a fallacy of composition.
"Perhaps it is the statistician in me, but I do not believe anything is random. Nothing occurs by random and nothing occurs by chance."
This is an appeal to authority.
"The Agile argument is based upon the idea that systematic study does not work for
software development. They believe 'most software is not predictable.'"
This is on the border line of the fallacy of "confusing cause and effect". Also this is a false dilemma.
"Every single time a development project is done, it is done differently. Documents are not cataloged and organized. There is no consistent usage of terms and symbols between projects, within projects, and even within single requirements
documents."
This is a distortion and thus a Straw Man argument.
Discussing pair programming Mr. Longstreet states, "The idea is that one programmer writes code and the other programmer stands over his shoulder and watches for mistakes."
This is a complete falsehood. He goes on to say, "I am not sure what problem pair programming is trying to solve. Most of the issues with software development are related to incomplete requirements, not coding."
The first part of his statements on pair programming is a Straw Man. Also his statement that most issues are related to incomplete requirements is confusing cause and effect, and is an appeal to consequences of a belief.
"Incomplete requirements are the biggest issue facing software development. I guess it is clear to the Agile folks that it is only logical to spend more time coding instead of cleaning up requirements or writing concise requirements in the first place."
As with many of the statements this one is of questionable cause and confusing cause and effect.
"They believe trial and error is the best method to gather and communicate requirements."
This is an appeal to ridicule, as is this statement:
"Agile proponents believe discipline is not necessary and inhibits productivity."
"Again the basic premise of Agile Methods is there is nothing I can do about my environment. I am a victim of my environment. I am a victim of circumstances. I can’t plan I can only react."
This is simply an example of poisoning the well. The fallacy goes like this:
- Unfavorable information (be it true or false) about person A is presented.
- Therefore any claims person A makes will be false.
I am only half way through this paper. I find it filled with falsehoods based upon poor research. Mr. Longstreet assumes an authoritative position at the beginning of his paper by stating his experience and declares that he specializes in non-biased scrutiny of people, processes, and businesses. He claims that multi-disciplinary study is a key component to his authority. Yet with these statements there is an obvious lack of research done concerning the current body of knowledge on the subjects of Agile and XP.
For those that are the authors of Agile methodologies I invite you to make concise statements defining your Agile Method. For Mr. Longstreet, I challenge him to clean up his paper and to cite sources and remove the unnecessary fallacies.
The topic of this posting will be "The Agile Method and Other Fairy Tales", by David Longstreet.
http://www.softwaremetrics.com/Agfa/Agile%20Paper.pdf
Mr. Longstreet gives an impressive background of travel and study of software organizations in many varied fields and marketplaces. I declares that he has "been dedicated to the idea of improving software productivity and quality...consulting and studying software organizations".
I have twenty three years experience in software development. I have always been involved with the improvement of software quality and productivity. My level of involvement is "applied", that is, I am a Computer Scientist and Software Engineer. I apply ideas for improvement and I stay around to live with the results. I have a vested interested in improving software development because those improvements directly affect me. I have developed software on the family dairy farm to manage accounts payable and production. I have delivered software for the Department of Energy to visualize magnetic fields, and to render 3D data visualizations from scanning/tunneling microscopes. I have written coding standards and guides which were adopted by the entire Computer Science department at the National Laboratory where I worked. I have delivered software for EMail applications, real-time stock market analysis, record managers for high performance indexing engines, eCommerce systems, new specialized GUI controls, charting packages, and many many other high quality and working systems. I can produce references if someone wants to question the veracity of my statements.
I have not limited myself to just studying the software industry. I am an award winning Dairy farmer and have been recognized for my techniques in raising dairy calves. I have traveled and performed voluntary service. I have studied Gothic Architecture and Renaissance Art, and I have traveled in Western Europe to see the actual works. I study languages and culture. I have done in-depth research on the Biblical Prophet Abraham and I currently study the Qur'an. I collect HO scale electric trains and I change the oil in my automobiles. I played the Cornet and Saxophone and I have proven myself to be a fairly good artist. Like everyone I have met, I am not a one-dimensional person.
I have developed software in 68000 and VAX assembly, Pascal, FORTAN 77, Ada, APL, Object C, Object Pascal, C, C++, Java, HyperTalk, C#, various scripting languages, and SQL stored procedures. I have managed teams of developers. I do not limit myself to my department either. Everyone knows that I will speak up in other departments and I am not afraid to go to the CEO or the Board. As a matter of fact, I have been demanding on many organizations and I have had no fear of any position.
I state all of this because Mr. Longstreet does so at the beginning of his paper and then later in the paper describes Agile users as one dimensional. Even with all of my experience there are many things to learn.
Like Mr. Longstreet, I was very skeptical of eXtreme Programming (XP) when I first heard of it. I started writing a paper, XP eXposed. As I studied XP and then applied parts of it I started to see what Kent Beck was describing. Soon I abandoned the paper and started applying parts of XP and I found value in XP.
I will now take some quotes from Longstreet's paper and make some comments:
"I have come to the conclusion that software developers cause most of their own problems. The root cause of most of the problems facing software development is actually caused by software developers themselves. They are creating their own complexity and chaos."
This argument is flawed and is known as an appeal to consequences of a belief.
"Agile methods want continuation and formal acceptance of the status quo... Up to this point in time software development has been a Wild West endeavor... IT has been sloppy. There is nothing new with Agile, because it only tries to formalize sloppiness."
This argument is flawed and is known as an appeal to ridicule, spite, and ultimately the argument turns to an appeal of tradition.
"I am bringing a level of professionalism and rigor to the software industry, and I hope you join me."
This is a setup for the well known fallacy known as an "appeal to authority".
"An Agile proponent will argue there is limited value in requirements specifications because the requirements are ever changing."
There may be someone that Mr. Longstreet views as an Agile proponent that has argued this point. That does not make it so. Use Cases and User Stories are the two approaches I am most familiar with and both are taught and used by Agile proponents. I personally believe that some of the confusion is with the roles of XP and the idea of Agile.
"'I think it's fair to say that customer practices are not addressed in Agile methods.' It is clear that understanding what the customer wants or helping the customer figure out what they want is not really part of Agile, and in turn not part of software development."
Mr. Longstreet is quoting someone from an online users group. Such methods of fact finding are laughable. This statement is a hasty generalization, and hearsay.
"It is clear that understanding what the customer wants or helping the customer figure out what they want is not really part of Agile, and in turn not part of software development."
I believe this is a fallacy of composition.
"Perhaps it is the statistician in me, but I do not believe anything is random. Nothing occurs by random and nothing occurs by chance."
This is an appeal to authority.
"The Agile argument is based upon the idea that systematic study does not work for
software development. They believe 'most software is not predictable.'"
This is on the border line of the fallacy of "confusing cause and effect". Also this is a false dilemma.
"Every single time a development project is done, it is done differently. Documents are not cataloged and organized. There is no consistent usage of terms and symbols between projects, within projects, and even within single requirements
documents."
This is a distortion and thus a Straw Man argument.
Discussing pair programming Mr. Longstreet states, "The idea is that one programmer writes code and the other programmer stands over his shoulder and watches for mistakes."
This is a complete falsehood. He goes on to say, "I am not sure what problem pair programming is trying to solve. Most of the issues with software development are related to incomplete requirements, not coding."
The first part of his statements on pair programming is a Straw Man. Also his statement that most issues are related to incomplete requirements is confusing cause and effect, and is an appeal to consequences of a belief.
"Incomplete requirements are the biggest issue facing software development. I guess it is clear to the Agile folks that it is only logical to spend more time coding instead of cleaning up requirements or writing concise requirements in the first place."
As with many of the statements this one is of questionable cause and confusing cause and effect.
"They believe trial and error is the best method to gather and communicate requirements."
This is an appeal to ridicule, as is this statement:
"Agile proponents believe discipline is not necessary and inhibits productivity."
"Again the basic premise of Agile Methods is there is nothing I can do about my environment. I am a victim of my environment. I am a victim of circumstances. I can’t plan I can only react."
This is simply an example of poisoning the well. The fallacy goes like this:
- Unfavorable information (be it true or false) about person A is presented.
- Therefore any claims person A makes will be false.
I am only half way through this paper. I find it filled with falsehoods based upon poor research. Mr. Longstreet assumes an authoritative position at the beginning of his paper by stating his experience and declares that he specializes in non-biased scrutiny of people, processes, and businesses. He claims that multi-disciplinary study is a key component to his authority. Yet with these statements there is an obvious lack of research done concerning the current body of knowledge on the subjects of Agile and XP.
For those that are the authors of Agile methodologies I invite you to make concise statements defining your Agile Method. For Mr. Longstreet, I challenge him to clean up his paper and to cite sources and remove the unnecessary fallacies.
Saturday, February 02, 2008
It's okay to THINK for Yourself
I started studying the Sofware Development process in 1985. In about four years I had learned techniques in estimation and learned the name of Halstead. I had learned of something called "COCOMO" and learned the name of Boehm. I learned of Jackson System Development and found out that Michael wasn't just the king of MoTown. I learned of the SW-CMM and learned the name of Humphrey. I learned of Brooks and Parnas, of Booch and many others.
The biggest thing I did not know at the time was that all of these topics were new. I came from a Dairy farm and I had never used a computer until college. I had seen a TRaSh 80 once before but that was it. As I took my math classes, physics classes, and computer science classes I just assumed that the C.S. stuff was "old" stuff and that everyone in industry used and believed in what I was learning in school. I had no idea that the topics were leading and even bleeding edge. My first programming language was Pascal on a VAX 11-750. I did not know that in industry at that same time many projects were done completely in assembly language and that in industry it is costly and takes time to switch from assembly to Pascal, ADA, or C.
My ignorance was a blessing in disguise. Since I assumed that everything I was taught was in common use in industry I had no problem analyzing the concepts as if they were "old" and in need of replacement. I was not mesmerized by these new concepts because I didn't know they were new.
I remember in my CS 327 class I was taught about the Waterfall model. I just went down stairs and got my old text book out. "Software Engineering: A Practitioner's approach. Second Edition" Chapter one of the text taught me the "classic life cycle" of software development. The text clearly states that the waterfall model was under criticism even at that time. It says that projects rarely follow the sequential flow, that customer's had difficulty stating all requirements explicitly, and that the customer had to be very patient because a working version of the software is not available until late in the project time span. The next section introduced Prototyping.
Well, I better fast forward to the now.
Now I have years of context. I know when XP and Agile came on the scene and I viewed them in their true position for the time as being new approaches. I have watched as the current set of software development practitioners beg to be told how to develop software. I mean just what I said. BEG TO BE TOLD. I have participated heavily in many online users groups, more lightly with local users groups, and heavily within the companies where I have worked. I have watched my peers look for someone to whom they can abdicate the decision of how to make good software.
In the late 80's and early 90's the companies I worked for brought in consultants which taught Waterfall, change control boards, and formal inspections. My peers accepted those methods as the way to develop software and seem to have never revisited that problem again. Well, we have moved on in our careers and now many are Managers, Directors, VP's, and even CXO's. Most have not revisited the problem of "how to develop software". I watch the next generation of software developers rage against the old machine. They want to develop software in better ways that work with the advancements in hardware and development tools. But they do what my generation did and abdicate the decision of how to make good software to the current set of consultants. This isn't totally bad unless they make the second mistake that many of my peers made and never revisit the question again.
I watch as people look for Agile Methods. Agile recipes is what they want. Secretly I think they are looking for guarantees. They want to say, "When I find myself in situation X I can apply practice Y and that is the best that can be done." Not only that, but they want to recite the authority of the practice as well, "Beck and Jeffries have both said this is how to do it." Well there you go, we could never question Beck or Jeffries now could we! :-)
It's great to read the latest books and articles, to talk with the authors, and even hire them to teach, clarify, and expound. Ultimately you will find yourself metaphorically alone and having to make decisions. At this point you can either think for yourself or you can blindly follow your method.
By current definitions of Agile Software development I probably practiced it for about a week. I immediately made changes to suit the needs I encountered. I moved on and I am still moving on. I never have cared if I was Agile or not. That was not the problem I was faced with. The problem I am faced with everyday is how to develop software the best I can in the current situation I find myself.
Is your problem whether or not you are Agile? Or is your problem developing software? If you recognize your problem to be developing software do your recognize what is causing the difficulty? If so then you can then say, "Will Agile practices address the issues I am faced with?" Think dog gone it. Think for yourself. Think and then apply yourself to the problems you face. Agile surely has many answers for many problems in Software development. Agile surely is not the answer all of the problems. And neither is CMMI, Lean, BDD, FDD, Spiral, Prototyping, or even DBU.
I have read countless threads on Agile users groups on how do you sell your group or company on Agile. I see this and I sigh. I think, here we go again. Think for a moment. Sell your group or company on solutions to real problems that are being faced. I advise on looking for root problems! For instance, if one of the problems is that the software doesn't meet the requirements when it is shipped then maybe you should institute rigorous reviews of check-ins and have a requirements sign off check list. OR, maybe the problem would be better addressed if there were incremental releases of completed features which are accepted by those that defined the requirement. Dig for the deeper problems and solve them first! Think! Think dog gone it. THINK! Think, because either answer may be right.
I have read countless threads on software development groups on the topic of "when should you optimize software". Often the posts are just a trap and have not disclosed the entire situation accurately or completely. That is, often the poster of the question has already identified that the software doesn't perform sufficiently and they are just waiting to have an argument. But if you are serious about the question then I say again, "Think!". If you are worried about scalability and user load then do you know which technologies are scalable? Do you know about load balancing? Do you know about distributed programming, or message queues? Do you know what about cluster? Virtual severs? Grid computing? If you are familiar with these subjects then you can at least say that we should develop our system to work behind an IP load balancer. These decisions must be made as soon as possible because they will effect how you develop the software. But if you understand such things as the Facade pattern then you can put off the decision until a bit later. But you have to know about alternatives and you have to think and apply.
If you are concerned with the runtime performance of a particular algorithm or function then you have to know the answer to the question of "what is fast enough?" Once you know that then you can exercise and profile the code to see if it is fast enough. Think about this! If you have the answer for how fast it has to be before you develop the algorithm it might guide you to the correct solution the first time instead of having to write and and then hope that it meets some unknown performance requirement. Think! That's what it's about, thinking. Maybe this suggestion is right for your situation and maybe it is not. Don't accept it blindly.
I have been told that I could not do a certain thing because the "agreed" upon software methodology doesn't allow one to do that. Bull pucky. People don't allow people to do things. Methodologies are neither alive nor can they hurt or heal. If that something needs to be done, and it is the right thing to do, then the problem must be real and identifiable. If it is all of these things then surely the problem and corresponding solution can be described. If it can be described then surely your peers can recognize the need or offer alternatives. It is not sufficient to dismiss the action on the basis of "the process doesn't allow for it." Think. THINK.
Do your best, and view your peers with an eye that they are smart and thinking people and that the burden lies upon you to communicate with them. If both sides of the conversation hold these values then both will try their best to hear what the other is saying. If you feel that you are the only one with those values then try and prove yourself wrong.
Think for yourself. It doesn't matter if the author of the latest and greatest software development methodology says you should do A and not do B. If you don't need to do A then don't do it. But know you don't need A for all of real reasons. Know why some people say you should do A and show how that doesn't apply. Don't follow for following's sake and don't be a contrar-ian for contrary's sake. Think. Learn. Share.
I try to do those things. I fail at it often, but I try.
Geoff
The biggest thing I did not know at the time was that all of these topics were new. I came from a Dairy farm and I had never used a computer until college. I had seen a TRaSh 80 once before but that was it. As I took my math classes, physics classes, and computer science classes I just assumed that the C.S. stuff was "old" stuff and that everyone in industry used and believed in what I was learning in school. I had no idea that the topics were leading and even bleeding edge. My first programming language was Pascal on a VAX 11-750. I did not know that in industry at that same time many projects were done completely in assembly language and that in industry it is costly and takes time to switch from assembly to Pascal, ADA, or C.
My ignorance was a blessing in disguise. Since I assumed that everything I was taught was in common use in industry I had no problem analyzing the concepts as if they were "old" and in need of replacement. I was not mesmerized by these new concepts because I didn't know they were new.
I remember in my CS 327 class I was taught about the Waterfall model. I just went down stairs and got my old text book out. "Software Engineering: A Practitioner's approach. Second Edition" Chapter one of the text taught me the "classic life cycle" of software development. The text clearly states that the waterfall model was under criticism even at that time. It says that projects rarely follow the sequential flow, that customer's had difficulty stating all requirements explicitly, and that the customer had to be very patient because a working version of the software is not available until late in the project time span. The next section introduced Prototyping.
Well, I better fast forward to the now.
Now I have years of context. I know when XP and Agile came on the scene and I viewed them in their true position for the time as being new approaches. I have watched as the current set of software development practitioners beg to be told how to develop software. I mean just what I said. BEG TO BE TOLD. I have participated heavily in many online users groups, more lightly with local users groups, and heavily within the companies where I have worked. I have watched my peers look for someone to whom they can abdicate the decision of how to make good software.
In the late 80's and early 90's the companies I worked for brought in consultants which taught Waterfall, change control boards, and formal inspections. My peers accepted those methods as the way to develop software and seem to have never revisited that problem again. Well, we have moved on in our careers and now many are Managers, Directors, VP's, and even CXO's. Most have not revisited the problem of "how to develop software". I watch the next generation of software developers rage against the old machine. They want to develop software in better ways that work with the advancements in hardware and development tools. But they do what my generation did and abdicate the decision of how to make good software to the current set of consultants. This isn't totally bad unless they make the second mistake that many of my peers made and never revisit the question again.
I watch as people look for Agile Methods. Agile recipes is what they want. Secretly I think they are looking for guarantees. They want to say, "When I find myself in situation X I can apply practice Y and that is the best that can be done." Not only that, but they want to recite the authority of the practice as well, "Beck and Jeffries have both said this is how to do it." Well there you go, we could never question Beck or Jeffries now could we! :-)
It's great to read the latest books and articles, to talk with the authors, and even hire them to teach, clarify, and expound. Ultimately you will find yourself metaphorically alone and having to make decisions. At this point you can either think for yourself or you can blindly follow your method.
By current definitions of Agile Software development I probably practiced it for about a week. I immediately made changes to suit the needs I encountered. I moved on and I am still moving on. I never have cared if I was Agile or not. That was not the problem I was faced with. The problem I am faced with everyday is how to develop software the best I can in the current situation I find myself.
Is your problem whether or not you are Agile? Or is your problem developing software? If you recognize your problem to be developing software do your recognize what is causing the difficulty? If so then you can then say, "Will Agile practices address the issues I am faced with?" Think dog gone it. Think for yourself. Think and then apply yourself to the problems you face. Agile surely has many answers for many problems in Software development. Agile surely is not the answer all of the problems. And neither is CMMI, Lean, BDD, FDD, Spiral, Prototyping, or even DBU.
I have read countless threads on Agile users groups on how do you sell your group or company on Agile. I see this and I sigh. I think, here we go again. Think for a moment. Sell your group or company on solutions to real problems that are being faced. I advise on looking for root problems! For instance, if one of the problems is that the software doesn't meet the requirements when it is shipped then maybe you should institute rigorous reviews of check-ins and have a requirements sign off check list. OR, maybe the problem would be better addressed if there were incremental releases of completed features which are accepted by those that defined the requirement. Dig for the deeper problems and solve them first! Think! Think dog gone it. THINK! Think, because either answer may be right.
I have read countless threads on software development groups on the topic of "when should you optimize software". Often the posts are just a trap and have not disclosed the entire situation accurately or completely. That is, often the poster of the question has already identified that the software doesn't perform sufficiently and they are just waiting to have an argument. But if you are serious about the question then I say again, "Think!". If you are worried about scalability and user load then do you know which technologies are scalable? Do you know about load balancing? Do you know about distributed programming, or message queues? Do you know what about cluster? Virtual severs? Grid computing? If you are familiar with these subjects then you can at least say that we should develop our system to work behind an IP load balancer. These decisions must be made as soon as possible because they will effect how you develop the software. But if you understand such things as the Facade pattern then you can put off the decision until a bit later. But you have to know about alternatives and you have to think and apply.
If you are concerned with the runtime performance of a particular algorithm or function then you have to know the answer to the question of "what is fast enough?" Once you know that then you can exercise and profile the code to see if it is fast enough. Think about this! If you have the answer for how fast it has to be before you develop the algorithm it might guide you to the correct solution the first time instead of having to write and and then hope that it meets some unknown performance requirement. Think! That's what it's about, thinking. Maybe this suggestion is right for your situation and maybe it is not. Don't accept it blindly.
I have been told that I could not do a certain thing because the "agreed" upon software methodology doesn't allow one to do that. Bull pucky. People don't allow people to do things. Methodologies are neither alive nor can they hurt or heal. If that something needs to be done, and it is the right thing to do, then the problem must be real and identifiable. If it is all of these things then surely the problem and corresponding solution can be described. If it can be described then surely your peers can recognize the need or offer alternatives. It is not sufficient to dismiss the action on the basis of "the process doesn't allow for it." Think. THINK.
Do your best, and view your peers with an eye that they are smart and thinking people and that the burden lies upon you to communicate with them. If both sides of the conversation hold these values then both will try their best to hear what the other is saying. If you feel that you are the only one with those values then try and prove yourself wrong.
Think for yourself. It doesn't matter if the author of the latest and greatest software development methodology says you should do A and not do B. If you don't need to do A then don't do it. But know you don't need A for all of real reasons. Know why some people say you should do A and show how that doesn't apply. Don't follow for following's sake and don't be a contrar-ian for contrary's sake. Think. Learn. Share.
I try to do those things. I fail at it often, but I try.
Geoff
Subscribe to:
Posts (Atom)