Friday, March 25, 2016
Saturday, March 12, 2016
Thursday, April 3, 2008
Seek small ideas – not big ones
As an organisation do you encourage your employees to suggest ideas for improving your business? If you don’t have a formal process by which ideas can be submitted and assessed, then chances are you are going to miss out on some very valuable information that is freely available.
Some recent research by Alan G. Robinson and Dean M. Schroeder which investigated the idea systems in over 150 companies in 17 countries came to the surprising conclusion that high performing companies go after lots of small ideas while low performing companies tend to go after big ones.
Why go after small ideas rather than big ones in an idea capture programme?
1. It can give your company a competitive advantage in the market place
Small ideas are more difficult to copy by competitors than big ones. Competitors may not be aware of small ideas where they will be aware of big initiatives (the grapevine of suppliers etc.)
Small ideas tend to be more company specific. Only work in the company they were generated in (e.g. Vidette Times – Indiana, USA (printing press rolls)
2. It can bring about performance excellence
Excellence is about getting details right it is about attention to detail. In a homogenous marketplace this is what separates companies from one other.
It is workers, not managers who can spot these inefficiencies
3. Small ideas lead to big things
Most big ideas start off small
Small ideas can help identify where there are problems that need to be addressed. Big problems frequently manifest themselves through a host of smaller signs or symptoms
When an idea is generated, ask
· Where else can this be used in the company?
· What are the patterns of the idea? I.e. are the small ideas all pointing toward a bigger problem?
4. It encourages employees and makes them feel heard
The research suggests that employees do not necessarily seek financial reward, and indeed offering it can cause problems such as who originally had the idea!
Setting up an idea capture programme
The difficulty with having many small ideas is that they can be difficult to capture and organise. Unless each idea if considered and processed, then employees will become disillusioned with the process. Client Solutions has developed an application that can help with the capture, review and approval of ideas from a multitude of sources. If this is something that would be of interest to you, then why not get in touch with us, we would love to show you what we have done in this space.
Some recent research by Alan G. Robinson and Dean M. Schroeder which investigated the idea systems in over 150 companies in 17 countries came to the surprising conclusion that high performing companies go after lots of small ideas while low performing companies tend to go after big ones.
Why go after small ideas rather than big ones in an idea capture programme?
1. It can give your company a competitive advantage in the market place
Small ideas are more difficult to copy by competitors than big ones. Competitors may not be aware of small ideas where they will be aware of big initiatives (the grapevine of suppliers etc.)
Small ideas tend to be more company specific. Only work in the company they were generated in (e.g. Vidette Times – Indiana, USA (printing press rolls)
2. It can bring about performance excellence
Excellence is about getting details right it is about attention to detail. In a homogenous marketplace this is what separates companies from one other.
It is workers, not managers who can spot these inefficiencies
3. Small ideas lead to big things
Most big ideas start off small
Small ideas can help identify where there are problems that need to be addressed. Big problems frequently manifest themselves through a host of smaller signs or symptoms
When an idea is generated, ask
· Where else can this be used in the company?
· What are the patterns of the idea? I.e. are the small ideas all pointing toward a bigger problem?
4. It encourages employees and makes them feel heard
The research suggests that employees do not necessarily seek financial reward, and indeed offering it can cause problems such as who originally had the idea!
Setting up an idea capture programme
The difficulty with having many small ideas is that they can be difficult to capture and organise. Unless each idea if considered and processed, then employees will become disillusioned with the process. Client Solutions has developed an application that can help with the capture, review and approval of ideas from a multitude of sources. If this is something that would be of interest to you, then why not get in touch with us, we would love to show you what we have done in this space.
Monday, March 10, 2008
A Truly Path-Based Approach
This month’s edition (March 2008) of the Harvard Business Review contains an article by David Upton and Bradley Staats called ‘Radically Simple IT’. It proposes an approach to implementing new IT systems that is preferable to the ‘big bang’ or ‘incremental’ methods favoured by most organisations.
The ‘Big Bang’ method involves spending a long period of time developing a system and then implementing it in one foul sweep. This clearly is problematic as the new system may end up being ‘legacy from the day it is turned on’. The incremental approach involves replacing an existing system one small piece at a time. This can take even longer than the ‘big bang’ approach and usually ends up duplicating what it replaces which prevents innovative advancement in the business.
Upton and Staat’s suggestion is one that has been used with particular success at a Japanese company called Shinsei. It is called the path-based approach. The idea behind the path based approach is to focus on providing a path for the new system to be developed over time rather than the functionality that will be in it. This approach takes into account the problems inherent in the implementation of new systems namely,
· People often cannot specify everything they will need at a project’s inception
· Unanticipated needs almost always arise once a system is in operation
· Persuading people to use and own the system after it is up and running is much easier said than done
Three foundational elements of the path-based approach are: -
1. Build a low-cost, efficient platform for running the company’s existing business
2. Ensure that platform is flexible enough to support the company’s growth into new areas (i.e. tweakable)
3. Forge together (don’t just align) business and IT
I was speaking recently with a man in South Africa who has used Serena TeamTrack to build such a foundation for his company. This bank operates in 38 countries round the world and employs about 40,000 people.
At his bank, over 30,000 users can log into TeamTrack to participate in over 300 different processes ranging from HR related processes to server commissioning to SLA management. In other words, TeamTrack extends into all parts of the business and links those parts together.
Continual Improvement is something that Shinsei also recognises as important. By using TeamTrack, companies are able to perform continual improvement even after a system is rolled out. The system can be tweaked extremely easily. Any changes that are needed can be implemented by configuration rather development. Business users can make changes themselves rather than having to renegotiate with software developers every time a change is needed.
So how does TeamTrack meet the three foundational elements of the path-based approach? TeamTrack provides a low cost framework (element 1) that can be easily modified to facilitate growth (element 2) by business people as well as IT people (element 3).
If you would like someone from Client Solutions to contact you to discuss how you might be able to leverage the power of TeamTrack in your organisation, please get in touch.
The ‘Big Bang’ method involves spending a long period of time developing a system and then implementing it in one foul sweep. This clearly is problematic as the new system may end up being ‘legacy from the day it is turned on’. The incremental approach involves replacing an existing system one small piece at a time. This can take even longer than the ‘big bang’ approach and usually ends up duplicating what it replaces which prevents innovative advancement in the business.
Upton and Staat’s suggestion is one that has been used with particular success at a Japanese company called Shinsei. It is called the path-based approach. The idea behind the path based approach is to focus on providing a path for the new system to be developed over time rather than the functionality that will be in it. This approach takes into account the problems inherent in the implementation of new systems namely,
· People often cannot specify everything they will need at a project’s inception
· Unanticipated needs almost always arise once a system is in operation
· Persuading people to use and own the system after it is up and running is much easier said than done
Three foundational elements of the path-based approach are: -
1. Build a low-cost, efficient platform for running the company’s existing business
2. Ensure that platform is flexible enough to support the company’s growth into new areas (i.e. tweakable)
3. Forge together (don’t just align) business and IT
I was speaking recently with a man in South Africa who has used Serena TeamTrack to build such a foundation for his company. This bank operates in 38 countries round the world and employs about 40,000 people.
At his bank, over 30,000 users can log into TeamTrack to participate in over 300 different processes ranging from HR related processes to server commissioning to SLA management. In other words, TeamTrack extends into all parts of the business and links those parts together.
Continual Improvement is something that Shinsei also recognises as important. By using TeamTrack, companies are able to perform continual improvement even after a system is rolled out. The system can be tweaked extremely easily. Any changes that are needed can be implemented by configuration rather development. Business users can make changes themselves rather than having to renegotiate with software developers every time a change is needed.
So how does TeamTrack meet the three foundational elements of the path-based approach? TeamTrack provides a low cost framework (element 1) that can be easily modified to facilitate growth (element 2) by business people as well as IT people (element 3).
If you would like someone from Client Solutions to contact you to discuss how you might be able to leverage the power of TeamTrack in your organisation, please get in touch.
Friday, December 7, 2007
The fourth project management constraint
Robert Graham, in an article titled ‘Beyond the Triple Constraints’ makes the case that project managers need to consider contribution to shareholder value as an additional variable. Anyone who has studied project management will be familiar with the triple constraints of time, quality and cost. Graham suggests that in the modern business environment focusing on these constraints alone is not enough.
He says that because project outcomes are increasingly vague (agile development methodologies and more fluid market place), costs are continually changing (caused by constantly changing outcomes) and schedules are more externally driven, an additional metric is needed.
A project manager who focuses exclusively on delivering a project in the quickest time to the highest quality and lowest cost may be doing a disservice to the organisation. Because the business environment is constantly changing, delivering on a plan that was agreed months or years before may not be the best outcome for the organisation now.
Suppose for example that a project manager was working on delivering a new product but during development some new technology emerged that made that project redundant. Continuing to work on that project would not be in the best interests of the organisation. Project managers need to consider more than just time, cost and quality. Graham suggests that an additional metric of shareholder value should be included. Increasing shareholder value is chosen of course because it should be the goal of any publically trading organisation (non-profit organisations aside). Project Managers need to constantly ask themselves, is my mix of projects maximising shareholder value given the way that the market is now; not the way it was a few months ago.
I agree with him that this should be the case but suggest that most project managers will be unable to ascertain which courses of action would best maximise shareholder value. Instead, project managers should focus on advancing strategic goals as communicated by senior management. It is the responsibility of senior management to communicate a small number of strategic goals that will lead to shareholder value maximisation, that all employees should be striving towards. This will result in a more coordinated, unified organisation and will avoid the danger where project managers take a course of action that increases shareholder value in the short term but damages its ability to do so in the long term.
Of course, for a project management office (PMO) to be able to take a real time look at its mix of projects requires a project and portfolio management tool. With such a tool, individuals within the organisation can take a holistic view of the PMO and ensure that it is constantly aligned with its strategic goals. If you would like to discuss how you can get that real time, holistic view of your PMO, please get in touch with us.
He says that because project outcomes are increasingly vague (agile development methodologies and more fluid market place), costs are continually changing (caused by constantly changing outcomes) and schedules are more externally driven, an additional metric is needed.
A project manager who focuses exclusively on delivering a project in the quickest time to the highest quality and lowest cost may be doing a disservice to the organisation. Because the business environment is constantly changing, delivering on a plan that was agreed months or years before may not be the best outcome for the organisation now.Suppose for example that a project manager was working on delivering a new product but during development some new technology emerged that made that project redundant. Continuing to work on that project would not be in the best interests of the organisation. Project managers need to consider more than just time, cost and quality. Graham suggests that an additional metric of shareholder value should be included. Increasing shareholder value is chosen of course because it should be the goal of any publically trading organisation (non-profit organisations aside). Project Managers need to constantly ask themselves, is my mix of projects maximising shareholder value given the way that the market is now; not the way it was a few months ago.
I agree with him that this should be the case but suggest that most project managers will be unable to ascertain which courses of action would best maximise shareholder value. Instead, project managers should focus on advancing strategic goals as communicated by senior management. It is the responsibility of senior management to communicate a small number of strategic goals that will lead to shareholder value maximisation, that all employees should be striving towards. This will result in a more coordinated, unified organisation and will avoid the danger where project managers take a course of action that increases shareholder value in the short term but damages its ability to do so in the long term.
Of course, for a project management office (PMO) to be able to take a real time look at its mix of projects requires a project and portfolio management tool. With such a tool, individuals within the organisation can take a holistic view of the PMO and ensure that it is constantly aligned with its strategic goals. If you would like to discuss how you can get that real time, holistic view of your PMO, please get in touch with us.
Thursday, November 22, 2007
The Long-Tail of Idea Generation
TeamTrack is used by companies around the world in a myriad of different ways. In fact Serena knows of over 600 different applications that have been created using TeamTrack and they estimate that there are hundreds if not thousands more that they do not know about.
In this series of articles we will look in detail at some of the applications that Client Solutions has delivered to customers in Ireland using Serena TeamTrack.
The Idea Forum Application
An article entitled ‘Big Results from Small Ideas’ by Alan G. Robinson and Dean M. Schroeder looks at the impact of idea generation schemes in companies. They examined over 150 companies in 17 countries and surprisingly found, that the companies that did best were those that implemented lots of small ideas rather than a few big ones. They said,
“We compared the best idea systems in the world--those implementing 20, 50 and even 100 ideas per employee per year--with medium- and low-performing systems. The purpose was to document what works to promote idea generation, what doesn't, and why. One of the most surprising findings of the ‘Ideas Are Free’ study was how high-performing companies focused on small ideas while low-performing companies tended to go after big ones.”
What we are talking about here is the long-tail of idea generation. It is better to focus on lots of small ideas than a couple of big ones. The value of all those small ideas combined together far exceeds the value of a few big ones. In the graph below, we can see that the combined value of the ‘big’ (green) ideas is less than the combined value of the ‘small’ (blue) ideas.
The difficulty with any long tail however, is that the amount of effort expended in gathering and evaluating each idea in the long-tail can outweigh the benefit to be derived from implementing them. That is why you need to automate the process.
Client Solution’s Idea Forum application allows employees throughout an organisation to submit innovative ideas that are then peer reviewed. The top ideas each month or quarter are escalated to management who will consider them for implementation.
The main advantage of this application is that it captures valuable innovative and creative suggestions throughout the organisation. By exposing them to other employees for consideration, the organisation ensures that the best ideas bubble to the top and are improved upon by a forum type discussion. The result is that creative energy is generated by all the ideas and comments on ideas, bouncing off each other.
One good idea can be more than enough to pay for the application and by giving those who are closest to your products or services a voice; who knows what other innovative opportunities can be harvested.
How does it work?
1. An idea is submitted into TeamTrack through a website on the company intranet
2. The author of the idea may take a few days to fine tune the details of the idea and attach images and various bits of documentation to it
3. When the author is ready to have it peer reviewed, she clicks the ‘Send for Peer Review’ button.
4. Users who have ‘peer review’ privileges, can browse the list of ideas that are available for review. They can click into an idea, read the associated documentation, add their own comments and then give the idea a rating.
5. The ideas with the highest average rating or number of reviews bubble to the top.
6. A manager moves all ideas out of the peer review state every quarter and the top ideas are put before management for consideration. The authors of the top rated ideas are rewarded in some way, usually by giving them a percentage of the savings or profit resulting from the idea being implemented.
If you would like to discuss implementing the Idea Forum application in your organisation then please get in touch.
In future articles I will be looking at some other Client Solutions applications developed in TeamTrack that Irish companies are capitalising on.
In this series of articles we will look in detail at some of the applications that Client Solutions has delivered to customers in Ireland using Serena TeamTrack.
The Idea Forum Application
An article entitled ‘Big Results from Small Ideas’ by Alan G. Robinson and Dean M. Schroeder looks at the impact of idea generation schemes in companies. They examined over 150 companies in 17 countries and surprisingly found, that the companies that did best were those that implemented lots of small ideas rather than a few big ones. They said,
“We compared the best idea systems in the world--those implementing 20, 50 and even 100 ideas per employee per year--with medium- and low-performing systems. The purpose was to document what works to promote idea generation, what doesn't, and why. One of the most surprising findings of the ‘Ideas Are Free’ study was how high-performing companies focused on small ideas while low-performing companies tended to go after big ones.”
What we are talking about here is the long-tail of idea generation. It is better to focus on lots of small ideas than a couple of big ones. The value of all those small ideas combined together far exceeds the value of a few big ones. In the graph below, we can see that the combined value of the ‘big’ (green) ideas is less than the combined value of the ‘small’ (blue) ideas.
The difficulty with any long tail however, is that the amount of effort expended in gathering and evaluating each idea in the long-tail can outweigh the benefit to be derived from implementing them. That is why you need to automate the process.
Client Solution’s Idea Forum application allows employees throughout an organisation to submit innovative ideas that are then peer reviewed. The top ideas each month or quarter are escalated to management who will consider them for implementation.
The main advantage of this application is that it captures valuable innovative and creative suggestions throughout the organisation. By exposing them to other employees for consideration, the organisation ensures that the best ideas bubble to the top and are improved upon by a forum type discussion. The result is that creative energy is generated by all the ideas and comments on ideas, bouncing off each other.
One good idea can be more than enough to pay for the application and by giving those who are closest to your products or services a voice; who knows what other innovative opportunities can be harvested.
How does it work?
1. An idea is submitted into TeamTrack through a website on the company intranet
2. The author of the idea may take a few days to fine tune the details of the idea and attach images and various bits of documentation to it
3. When the author is ready to have it peer reviewed, she clicks the ‘Send for Peer Review’ button.
4. Users who have ‘peer review’ privileges, can browse the list of ideas that are available for review. They can click into an idea, read the associated documentation, add their own comments and then give the idea a rating.
5. The ideas with the highest average rating or number of reviews bubble to the top.
6. A manager moves all ideas out of the peer review state every quarter and the top ideas are put before management for consideration. The authors of the top rated ideas are rewarded in some way, usually by giving them a percentage of the savings or profit resulting from the idea being implemented.
If you would like to discuss implementing the Idea Forum application in your organisation then please get in touch.
In future articles I will be looking at some other Client Solutions applications developed in TeamTrack that Irish companies are capitalising on.
Friday, November 16, 2007
Using Monte Carlo Simulation to derive more accurate project duration estimates
Monte Carlo Simulation is a relatively new technique in the field of mathematics that uses simulation rather than precise analytical calculations to arrive at an estimated value. In essence it introduces noise into a mathematical analysis. In the real world, actual outcomes seldom match predicted outcomes that are based on probabilities. ‘Noise’ or unanticipated influences often encroach on a situation, causing spurious results.
In this article I am going to look at how Monte Carlo Simulation could be used to give more accurate project duration estimates.
No project runs entirely uninterrupted from start to finish. Team members become ill or other higher priority tasks arise and require immediate attention. This is the ‘noise’ that I am referring to. ‘Noise’ is basically all the things that interfere with the normal running of a project. By looking back at the accuracy of a team’s estimates in the past and injecting random noise, the hope is that a more accurate estimate can be attained.
Forecasting how long a project is going to take has been a near impossible task for companies for years. Few large infrastructure projects where there is a team and strict deadlines in place, are completed on time.
Hugh W. Ryan, in an article for Outlook Journal, summarized research that showed:
- Only 8 percent of applications projects costing between $6 million and $10 million succeed.
- Among all IT development projects, only 16% are delivered to acceptable cost, time and quality.
- Cost overruns for IT projects have been estimated at $59 billion in the United States alone.
According to the Economist, a £456m ($844m) project for Britain's Child Support Agency came in over a year late, and has failed to deliver payments to more than half of eligible applicants and according to Cuppan approximately 80% of the medium to large software projects executed within the Department of Defense (DoD) were 100% over budget and 90% were at least one year behind schedule.
As the main cost in software development is developers, time over-runs quickly translate into cost over-runs. The accurate calculation of how long a project is going to take is vital to the success of a project. With a team of 20 developers, for example, that is costing €30,000 to €40,000 per week, delays soon add up.
In looking for an example of a large project that has run way over original estimation, consider the repeated delays of Microsoft’s Vista operating system. Vista was intended to replace Windows XP as Microsoft’s main operating system. While it is ambitious in terms of what it hoped to achieve (new file system structures and interface standards), it has become something of a joke in the industry. Every few months, Microsoft came out with a new vague release date intention which was ultimately not met.
When asked about Vista during an interview in Sept 2003, Microsoft Group Vice President Jim Allchin was similarly vague. "It's all a question of probabilities," he said. "[2005 is] our target. But there's a probability it may make it, it may not ... The truth is, these are targets ... We'll know so much more when we hit Beta 1. And we're not going to be at Beta 1 at the PDC [Professional Developers Conference in late October]. Once we hit Beta 1, we'll be able to get customer feedback. You can't predict when a product is going to ship until you get some customer feedback."
Almost two years later, there was still no release date for the new operating system. What are the difficulties in estimating a release date for software?
Difficulties with Software Project Duration Estimation
1. One of the main problems associated with estimating software project duration is defining what is included in the product offering and what is excluded. As specifications are fleshed out and then agreed upon, there is a tendency for management to add to, or change requirements as customer feedback is received. This can clearly be seen from Jim Allchin’s comments in the quote above where he observes that customer feedback will alter the proposed schedule.
2. Boehm and Fairley mention the lack of clarity when defining what work is included in a software project’s definition. They say
‘Does an estimate of 100 person-months for “software development” include analysis and design, integration and test, deployment, management, or uncompensated overtime? If you use the estimate without knowing the answers [to these questions], you can get yourself into serious trouble’
Projects are often embarked upon without a clear understanding of exactly what any declared deadlines relate to.
3. Design documents are often not detailed enough, so that when it comes to the build phase of the project, there are too many unknowns still in existence. Tasks that were assumed to be straightforward turn out to be more complex and so take much longer to develop than was anticipated. In fact Armour describes the software developer’s role as
4. The level of maturity of the software development process in an organisation or team is another large factor. The more experience a team has in developing software, the more efficient it will become.
5. Another difficulty is the fact that most developers, when asked how long a task will take, tend to use very inaccurate ‘guestimations’. Developers are notoriously bad at estimating how long a task will take. There is a problem as Armour notes,
‘large commitments of lots of money are made by big companies based on hunches and guesses by junior programmers.’
6. Kolmogorov complexity, a well-established and accepted area of information theory and computer science, describes the inherent complexity of software. In effect it claims that there is no algorithm for finding the shortest program with a desired behaviour – or there is no best way of doing a project. When we accept that there is no best way of writing a piece of software, we realise that project length estimation cannot be done with 100% accuracy.
7. An application of the second law of thermodynamics is also relevant here. As there are far more things that will slow down a project than will speed it up, we almost always underestimate.
When one considers all these reasons why software project estimation is problematic the question must be asked, are there any techniques for maximising the accuracy of estimates. It is not good enough to dismiss it as an impossible task. Companies that invest in software development insist upon some sort of estimate so as to be able to plan budgets and schedules.
Current Techniques for estimating project length
The following are some techniques that are used by industry to improve estimates.
1. Guess
"Software is a manufacturing process and manufacturing isn't a guessing game. There is too much at stake," (Boz Elloy, senior vice president of software products at Borland Software). This is obviously the most inaccurate method of estimation. It uses the expectation that a developer or team of developers will have a good ‘feeling’ for the size and expected duration of a project based on their experience in past projects. One of the obvious difficulties is the amount of unknowns. Armour in his article ‘Ten Unmyths of Project Estimation’, stresses that a developer’s primary job is not to write code but to gain knowledge. He suggests that because ‘much of software development involves acquiring knowledge we do not already have, we can neither accurately estimate how long it will take us or even if we have got it all.’
A way of simplifying this guess is to break the work into smaller more manageable pieces and estimate how long each piece will take. This technique called modelling assumes however a good understanding of what the various elements of a project are. This is not always the case.
2. Improved Guessing – Modelling
Karl Wiegers once said, ‘If a picture is worth a thousand words, a model is worth 1024 lines of code’ (ouch!) Modelling is a very widely used technique that development teams use to get a better understanding of the various elements of a project and as a result can better estimate the amount of time needed to complete it. There are various techniques that are available for modelling including the Unified Modelling Language (UML). This is a standard that allows developers to describe the way parts of a program flow into each other and are connected. Although it does not focus primarily on the length of time each element will take to develop, it is the most popular model and gives a very good understanding of the overall product. Once these elements are identified, they can each be further broken down until the number of unknowns is reduced to a manageable size and better estimates can be made. According to Davis
‘A model simply provides us with a richer, higher level, and more semantically precise set of constructs than the underlying natural language. Using such a model reduces ambiguity, makes it easier to check for incompleteness and may at times improve understandibility.’
3. Improved Guessing – Benchmarking
This approach involves looking backwards in order to look forwards. When a team embarks on the process of estimating how long each element in a project will take, they will make many mistakes. Benchmarking involves looking back at how a software team has performed relative to its estimates in the past. This is the most accurate method mentioned thus far.
Software developers record what their estimates are prior to embarking on the project or task and then record the actual time it took. This has two main benefits. First it teaches developers how to estimate. Estimating is a skill that needs to be developed. By sitting down with a developer and reviewing his or her estimates, their accuracy can be increased over time.
The second benefit in recording estimates and actuals is that the success rate can be used in the future against future estimates. Furthermore, if a manager knows how long a certain task took in the past then a similar task should take a similar amount of time in the future. If a developer tends to be overly optimistic in his estimates by a factor of X then it is likely that he will do so again in the future. By recording the developer’s performance and success rate, future estimates can be more accurate.
A word of warning however. As each project is different, the length of time to perform the tasks in it will be different. Our ability to acquire knowledge we do not have however, tends to be similar, no matter what that knowledge might be.
Monte Carlo Simulation
Monte Carlo Simulation is a relatively new technique in the field of mathematics that uses simulation rather than precise analytical calculations to arrive at an estimated value. In essence it introduces noise into a mathematical analysis. In the real world, actual outcomes seldom match predicted outcomes that are based on probabilities. ‘Noise’ or unanticipated influences often encroach on a situation, causing spurious results.
Software development is a process where, as we have seen above, there are many factors that can influence the amount of time a project will take. As each project is different in terms of goals, environment and personnel (customers and team) the amount of ‘noise’ exerted on a project schedule is phenomenal.
The following is an application of Monte Carlo Simulation that may be used to allow the project manager better calculate a delivery date for a project. This is my own proposal that I believe is better for calculating an estimate as it factors in the fact that there are so many unknowns in software development. The steps are as follows: -
1. Get your team to estimate how long a task will take and then record the actual amount of time that task took. This gives us a series of estimates vs. actual for each team member.
2. Determine how far out each estimate is and express it as a percentage. For example, if an estimate was for 20 days and the actual was 30 days, then record that the estimate was 50% out.
3. Determine the frequency that each deviation occurred and calculate the probability that that deviation will occur again. Also work out a value between 0 and 99 that corresponds to that deviation. This value will be used later on.
4. For the next project that we are about to embark on, get Peter to estimate how long each task will take.
5. In this step we are going to inject some noise into Peter’s estimates. Noise by its nature is random so we get a series of random numbers from a random number generator.
6. We take each random number in turn and compare it with a value in the 4th column of step 3. This will give us an expected deviation and from that we can calculate the time differential.
For example, we take the first random number 39. By looking at the table in step 3 we see that 39 corresponds to a deviation of 21% -> 40%. We take Peter’s estimate of 9 days and combine it with the deviation and estimate that if this level of noise occurred, we would expect Peter to finish the task between 10.89 days and 12.6 days.
9 + (21% of 9 days) = 10.89 days
9 + (40% of 9 days) = 12.6 days
For each task we will add three different levels of noise and see what the best and worst case for Peter doing the task are.
We work out therefore that for Peter’s estimate of 9 days and given the level of accuracy he has shown in the past in his estimates, we should expect him to finish the task between 9.99 and 12.6 days. If we calculate out his other estimates we find the following
For the entire project we come up with the following estimates
Conclusion
While this method isn’t perfect, it moves beyond guessing and benchmarking to account of the noise that enters into all projects. Of course all the calculations and tables can be automated out so that its complexity is hidden from the user. I believe that it will give more accurate estimates however that organisations can use to improve their project estimations.
Random deviations will occur with the same frequency that they occurred in the past.
In this article I am going to look at how Monte Carlo Simulation could be used to give more accurate project duration estimates.
No project runs entirely uninterrupted from start to finish. Team members become ill or other higher priority tasks arise and require immediate attention. This is the ‘noise’ that I am referring to. ‘Noise’ is basically all the things that interfere with the normal running of a project. By looking back at the accuracy of a team’s estimates in the past and injecting random noise, the hope is that a more accurate estimate can be attained.
Forecasting how long a project is going to take has been a near impossible task for companies for years. Few large infrastructure projects where there is a team and strict deadlines in place, are completed on time.
Hugh W. Ryan, in an article for Outlook Journal, summarized research that showed:
- Only 8 percent of applications projects costing between $6 million and $10 million succeed.
- Among all IT development projects, only 16% are delivered to acceptable cost, time and quality.
- Cost overruns for IT projects have been estimated at $59 billion in the United States alone.
According to the Economist, a £456m ($844m) project for Britain's Child Support Agency came in over a year late, and has failed to deliver payments to more than half of eligible applicants and according to Cuppan approximately 80% of the medium to large software projects executed within the Department of Defense (DoD) were 100% over budget and 90% were at least one year behind schedule.
As the main cost in software development is developers, time over-runs quickly translate into cost over-runs. The accurate calculation of how long a project is going to take is vital to the success of a project. With a team of 20 developers, for example, that is costing €30,000 to €40,000 per week, delays soon add up.
In looking for an example of a large project that has run way over original estimation, consider the repeated delays of Microsoft’s Vista operating system. Vista was intended to replace Windows XP as Microsoft’s main operating system. While it is ambitious in terms of what it hoped to achieve (new file system structures and interface standards), it has become something of a joke in the industry. Every few months, Microsoft came out with a new vague release date intention which was ultimately not met.
When asked about Vista during an interview in Sept 2003, Microsoft Group Vice President Jim Allchin was similarly vague. "It's all a question of probabilities," he said. "[2005 is] our target. But there's a probability it may make it, it may not ... The truth is, these are targets ... We'll know so much more when we hit Beta 1. And we're not going to be at Beta 1 at the PDC [Professional Developers Conference in late October]. Once we hit Beta 1, we'll be able to get customer feedback. You can't predict when a product is going to ship until you get some customer feedback."
Almost two years later, there was still no release date for the new operating system. What are the difficulties in estimating a release date for software?
Difficulties with Software Project Duration Estimation
1. One of the main problems associated with estimating software project duration is defining what is included in the product offering and what is excluded. As specifications are fleshed out and then agreed upon, there is a tendency for management to add to, or change requirements as customer feedback is received. This can clearly be seen from Jim Allchin’s comments in the quote above where he observes that customer feedback will alter the proposed schedule.
2. Boehm and Fairley mention the lack of clarity when defining what work is included in a software project’s definition. They say
‘Does an estimate of 100 person-months for “software development” include analysis and design, integration and test, deployment, management, or uncompensated overtime? If you use the estimate without knowing the answers [to these questions], you can get yourself into serious trouble’
Projects are often embarked upon without a clear understanding of exactly what any declared deadlines relate to.
3. Design documents are often not detailed enough, so that when it comes to the build phase of the project, there are too many unknowns still in existence. Tasks that were assumed to be straightforward turn out to be more complex and so take much longer to develop than was anticipated. In fact Armour describes the software developer’s role as
4. The level of maturity of the software development process in an organisation or team is another large factor. The more experience a team has in developing software, the more efficient it will become.
5. Another difficulty is the fact that most developers, when asked how long a task will take, tend to use very inaccurate ‘guestimations’. Developers are notoriously bad at estimating how long a task will take. There is a problem as Armour notes,
‘large commitments of lots of money are made by big companies based on hunches and guesses by junior programmers.’
6. Kolmogorov complexity, a well-established and accepted area of information theory and computer science, describes the inherent complexity of software. In effect it claims that there is no algorithm for finding the shortest program with a desired behaviour – or there is no best way of doing a project. When we accept that there is no best way of writing a piece of software, we realise that project length estimation cannot be done with 100% accuracy.
7. An application of the second law of thermodynamics is also relevant here. As there are far more things that will slow down a project than will speed it up, we almost always underestimate.
When one considers all these reasons why software project estimation is problematic the question must be asked, are there any techniques for maximising the accuracy of estimates. It is not good enough to dismiss it as an impossible task. Companies that invest in software development insist upon some sort of estimate so as to be able to plan budgets and schedules.
Current Techniques for estimating project length
The following are some techniques that are used by industry to improve estimates.
1. Guess
"Software is a manufacturing process and manufacturing isn't a guessing game. There is too much at stake," (Boz Elloy, senior vice president of software products at Borland Software). This is obviously the most inaccurate method of estimation. It uses the expectation that a developer or team of developers will have a good ‘feeling’ for the size and expected duration of a project based on their experience in past projects. One of the obvious difficulties is the amount of unknowns. Armour in his article ‘Ten Unmyths of Project Estimation’, stresses that a developer’s primary job is not to write code but to gain knowledge. He suggests that because ‘much of software development involves acquiring knowledge we do not already have, we can neither accurately estimate how long it will take us or even if we have got it all.’
A way of simplifying this guess is to break the work into smaller more manageable pieces and estimate how long each piece will take. This technique called modelling assumes however a good understanding of what the various elements of a project are. This is not always the case.
2. Improved Guessing – Modelling
Karl Wiegers once said, ‘If a picture is worth a thousand words, a model is worth 1024 lines of code’ (ouch!) Modelling is a very widely used technique that development teams use to get a better understanding of the various elements of a project and as a result can better estimate the amount of time needed to complete it. There are various techniques that are available for modelling including the Unified Modelling Language (UML). This is a standard that allows developers to describe the way parts of a program flow into each other and are connected. Although it does not focus primarily on the length of time each element will take to develop, it is the most popular model and gives a very good understanding of the overall product. Once these elements are identified, they can each be further broken down until the number of unknowns is reduced to a manageable size and better estimates can be made. According to Davis
‘A model simply provides us with a richer, higher level, and more semantically precise set of constructs than the underlying natural language. Using such a model reduces ambiguity, makes it easier to check for incompleteness and may at times improve understandibility.’
3. Improved Guessing – Benchmarking
This approach involves looking backwards in order to look forwards. When a team embarks on the process of estimating how long each element in a project will take, they will make many mistakes. Benchmarking involves looking back at how a software team has performed relative to its estimates in the past. This is the most accurate method mentioned thus far.
Software developers record what their estimates are prior to embarking on the project or task and then record the actual time it took. This has two main benefits. First it teaches developers how to estimate. Estimating is a skill that needs to be developed. By sitting down with a developer and reviewing his or her estimates, their accuracy can be increased over time.
The second benefit in recording estimates and actuals is that the success rate can be used in the future against future estimates. Furthermore, if a manager knows how long a certain task took in the past then a similar task should take a similar amount of time in the future. If a developer tends to be overly optimistic in his estimates by a factor of X then it is likely that he will do so again in the future. By recording the developer’s performance and success rate, future estimates can be more accurate.
A word of warning however. As each project is different, the length of time to perform the tasks in it will be different. Our ability to acquire knowledge we do not have however, tends to be similar, no matter what that knowledge might be.
Monte Carlo Simulation
Monte Carlo Simulation is a relatively new technique in the field of mathematics that uses simulation rather than precise analytical calculations to arrive at an estimated value. In essence it introduces noise into a mathematical analysis. In the real world, actual outcomes seldom match predicted outcomes that are based on probabilities. ‘Noise’ or unanticipated influences often encroach on a situation, causing spurious results.
Software development is a process where, as we have seen above, there are many factors that can influence the amount of time a project will take. As each project is different in terms of goals, environment and personnel (customers and team) the amount of ‘noise’ exerted on a project schedule is phenomenal.
The following is an application of Monte Carlo Simulation that may be used to allow the project manager better calculate a delivery date for a project. This is my own proposal that I believe is better for calculating an estimate as it factors in the fact that there are so many unknowns in software development. The steps are as follows: -
1. Get your team to estimate how long a task will take and then record the actual amount of time that task took. This gives us a series of estimates vs. actual for each team member.
2. Determine how far out each estimate is and express it as a percentage. For example, if an estimate was for 20 days and the actual was 30 days, then record that the estimate was 50% out.
3. Determine the frequency that each deviation occurred and calculate the probability that that deviation will occur again. Also work out a value between 0 and 99 that corresponds to that deviation. This value will be used later on.
4. For the next project that we are about to embark on, get Peter to estimate how long each task will take.
5. In this step we are going to inject some noise into Peter’s estimates. Noise by its nature is random so we get a series of random numbers from a random number generator.
6. We take each random number in turn and compare it with a value in the 4th column of step 3. This will give us an expected deviation and from that we can calculate the time differential.
For example, we take the first random number 39. By looking at the table in step 3 we see that 39 corresponds to a deviation of 21% -> 40%. We take Peter’s estimate of 9 days and combine it with the deviation and estimate that if this level of noise occurred, we would expect Peter to finish the task between 10.89 days and 12.6 days.
9 + (21% of 9 days) = 10.89 days
9 + (40% of 9 days) = 12.6 days
For each task we will add three different levels of noise and see what the best and worst case for Peter doing the task are.
We work out therefore that for Peter’s estimate of 9 days and given the level of accuracy he has shown in the past in his estimates, we should expect him to finish the task between 9.99 and 12.6 days. If we calculate out his other estimates we find the following
For the entire project we come up with the following estimates
Conclusion
While this method isn’t perfect, it moves beyond guessing and benchmarking to account of the noise that enters into all projects. Of course all the calculations and tables can be automated out so that its complexity is hidden from the user. I believe that it will give more accurate estimates however that organisations can use to improve their project estimations.
Random deviations will occur with the same frequency that they occurred in the past.
Subscribe to:
Posts (Atom)