The software industry has had to deal with the issue of software estimation for years. Unfortunately, there are many different opinions on how software development estimation should be handled, and there is no common understanding of what works best. This post discusses why software estimations often fail and provides advice on how organizations can use them more effectively to make better decisions before the planning phase of projects
Free software project estimate template
DownloadThe aim of this essay is to explain how to use estimation before a project begins. We don't address techniques for projects that are already underway.
A software development estimation should be an unbiased analytical process. Project estimation aims to understand the required effort and time to complete a project or task. This information should be used to make better decisions about direction.
It is very difficult to make a vigorous, plausible, and job-risking defence of an estimate that is derived by no quantitative method, supported by little data, and certified chiefly by the hunches of the managers. — Fred Brooks
Software estimates are not a target. They are not a commitment or a goal that needs to be met. Project estimates are not a quote; they are analytical judgments, not negotiation tools. Using software project estimates as a tender offer and selecting the smallest is a typical executive's antipattern. It should not be used for this purpose.
Most of the predictions are incorrect, and the problem is not neutral. According to industry statistics, the software development business is biased toward underestimation.
Using the tiniest estimate's size as a reason for selecting a partner for the next software development project merely results in making a decision based on incorrect information. There are many options to pick from before the software development project begins, ranging from trade-offs over features even to project cancellation if the most realistic estimate isn't acceptable. If the predicted budget has already been wasted by lousy forecasting, it will be too late to make such judgments.
No plan survives contact with the enemy. – Field Marshal Helmuth Graf von Moltke
Numerous pieces of research have been conducted during decades of software development. Let's go through them one by one in chronological order.
The cult book by Fred Brooks, released in 1974, identifies famous and fundamental Brook's law, according to which "adding manpower to a late software project makes it later." He also says he has a rule of thumb for successfully scheduling the project: planning 1/3, coding 1/6, and testing 1/2. According to the book, only the coding part often has accurate estimates, comprising 1/6 of the whole project. Of course, it's for the outdated heavy Waterfall software development life cycle, but many organizations still use it. On the other hand, Modern Agile project management does not promise a quicker software development process; instead, it encourages development teams to focus on critical features and tiny releases while reducing waste.
According to fundamental work published in 1992 by Albert L. Lederer and Jayesh Prasad, approximately two-thirds of all projects significantly exceed their estimates.
The Standish Group has published yearly reports on software engineering projects' success and performance since 1994. The first study caused a stir in the software engineering world: only 16% of software engineering projects were completed on time and within budget. At the same time, over half were underestimated, and nearly one-third completely failed and were abandoned. The average cost estimation overrun was approximately 100%, and the average project schedule overrun was 120%. But the accuracy is undoubtedly worse than those figures suggest. The Standish Group discovered that late projects frequently discarded a lot of functionality to fulfill their deadlines and budgets.
We see 30% of successful and 20% of completely failed projects in the CHAOS Report 2015. It's incredible: 70% of all software projects fail to satisfy expectations. However, this is not all; according to the study, only 11% of Waterfall projects were successful, with a 29% failure rate, which nearly verifies the 1994 research. Conversely, we have Agile projects with a success rate of 40% and a failure rate of 9%. Imagine if software development processes that don't even require an upfront project estimation process perform almost four times better.
The fascinating thing is that both reports stress the importance of user and executive management engagement. The essential success factor in 1994, the presence of a clear work breakdown structure, was eliminated in 2015. Software developers and project managers sought to work with uncertain and changing requirements using Agile to tackle this problem.
Steve McConnell, the author of a software estimation bestseller published in 2006, offers fascinating data from one company.
A diagonal line symbolizes perfect scheduling accuracy. The graph should ideally show data points clustering closely around the line. Instead, the overwhelming majority of the data points displayed are above the line, indicating project overruns. "The common definition of estimate is the most optimistic prediction that has a non-zero probability of coming true." - Steve McConnell refers to Tom DeMarco's "Waltzing with Bears," another software estimation bestseller.
One of the book's central themes is that targets can only be achieved on controlled projects. Removing non-essential features, redefining criteria, replacing less-experienced personnel with more-experienced people, and so on are typical project control activities. To do so, the project manager must be able to alter the project to match the estimate.
According to a 2012 study from the global consulting firm McKinsey, software development projects have an average cost overrun of 66%, a timetable delay of 33%, and a benefits shortcoming of 17%. Furthermore, they discovered that overall, the investigated IT projects had a cost overrun of $66 billion, more than the GDP of Luxembourg.
What if the reason for missed expectations is poor formal project management framework and estimating skills? Let's look at NASA, which indeed uses excellent practices and has good estimating abilities and all the possible software estimation tools. The Government Accountability Office (GAO) has conducted an annual review of the status of NASA's key projects. According to the researchers, the weighted average cost increase was up to 47 percent. In 2004, the Congressional Budget Office reviewed the first and revised budgets of 72 NASA projects stretching from 1977 to 2000 in order to compare them. The original budgets for these initiatives were $41.1 billion, whereas the amended budgets were $66.3 billion, a 61 percent increase.
I'm unsure if NASA's exact cost overrun is relevant data for a typical software project. The why behind the increase is far more fascinating than its actual progress. NASA has never found an answer to the initial overrun problem since the 1970s; they've simply discovered new causes.
The causes are as follows: faulty definitions before the budget were made, overly optimistic cost predictions, inability to execute the initial schedule, technical difficulties, and work breakdown structure changes. These reasons are generally seen in each project and at every company.
There's no need to waste time and money attempting to fight these issues; far better to accept them, minimize damage, and turn them into advantages.
What is an acceptable estimate when estimates are frequently incorrect, and we can't eliminate the causes of error? It should be honest, accurate enough, and precise enough to enable a decision to be made.
If an estimate isn't honest and made to match the executive's preconceived notion, it's pointless and harmful. It does not let you decide; instead, it supports a false one.
The decision will be incorrect if the estimate is inaccurate and the error is measured in many orders of magnitude. The primary objective is to make an accurate call regarding whether or not to start a project. A reasonable estimate also may help determine which features are not essential to complete to fulfill a financial goal. The first estimate should not be relied on to decide what to do simply because the project requirements will alter.
If an estimate is not precise enough, and if the range is too broad, it cannot be used to make a decision owing to high uncertainty. Therefore, such an estimate is of no use.
But how will the executive know whether or not the estimate is sufficient?
We can't read minds, but businesses cannot influence honest predictions. An honest estimator will always be willing to cut the project scope to fit the budget but never offer to finish features in fewer hours.
McConnell offers a simple test to evaluate software estimation abilities in his book. Fill in the upper and lower bounds that, in your opinion, give you a 90% chance of containing the correct value for each question. Please do not search for answers—the purpose of this quiz is to test your forecasting abilities, not your research ability. The answers will be listed at the end of this article.
The average number of correct answers for the test takers is 2.8. Only 2 percent of quiz takers get 8 or more answers right. Most people's intuitive feeling of "90 percent confident" is comparable to "30 percent confident."
Instead of improving our guessing skills, eliminating guessing as much as possible is much more effective. We could count and then calculate instead of guessing. We don't need to have complex tools; the simple formula will help:
Effort Estimation = NumberOfRequirements * AverageEffortPerRequirement
If we have more than ten requirements in a project, the law of large numbers is effective, and estimating each one is unnecessary. If the software development project isn't the first for the organization, historical data is already available to determine the average number of hours per requirement.
Of course, some activities may require much more time than the average job. Having a previous history of similar past projects would be fantastic.
If we don't have enough historical data and the task cannot be characterized as "average," we may utilize a guess. Using a PERT formula, a three-point estimate is a good practice in this situation:
PERT = (OptimisticEffort + 4*MostLikelyEffort + PessimisticEffort) / 6
Three points are an optimistic scenario, when we anticipate that everything will work out; a most likely scenario, when something can go wrong; and a pessimistic case, when everything is certain to fail. Of course, we must keep in mind the typical estimating quiz result.
It's important to remember that software development estimation is never a single figure; it's a range of possibilities. If we use PERT, the range should be determined utilizing standard deviation. However, historical error data from similar projects is required to calibrate this deviation, making the software development estimation process more complex.
In 1981, Barry Boehm created the first version of what Steve McConnell dubbed the "cone of uncertainty." This software estimation technique will assist us in determining what range can estimate various phases of the software development life cycle.
It's critical to note that the Cone of Uncertainty shows the maximum probable accuracy of all estimation practices at various project phases. The Cone is a graph that depicts the error in estimates produced by skilled estimators. It's easy to do much worse. There's no way to be more correct; all you can do is hope for better luck.
Another essential thing to recognize is that an actual estimate may never improve if the project management is not adequate or if the estimators aren't skilled. When the focus of a project isn't on reducing uncertainty — it's not a Cone but rather a Cloud that lingers until the end of the project.
What does it imply for an executive to review an estimate? Assume that an estimate is frequently provided before the requirements are finished. Because requirements are known to alter throughout the course of a project, there's no need to nail them down precisely. The Cone states that the best achievable precision in this project stage is 0.67x to 1.5x. It means that if the upper bound isn't at least 2.24 times larger than the lower bound, the estimate isn't accurate enough or isn't fair.
How should we define requirements if we know they will change throughout the project? There is a fantastic agile software development technique known as the user story. A user story is not even a formal requirement; instead, it's a promise to communicate later. For example: "A user can add a product to the cart" or "A user may pay for an order." The word "INVEST" is a popular acronym that signifies how a user story should be created.
Independent. It should be feasible to rearrange the order of stories.
Valuable. It should be obvious why the user story is needed.
Estimable. It should be feasible to estimate each user story.
Small. It should be feasible to create many stories in one iteration.
Testable. Whether the user story is complete or not should be evident by performing the user's action.
Here are some of the most common blunders that often result in underestimates.
Is it true that all of the project's goals have been satisfied by user stories? Before estimating software projects, it's a good idea to write up a project goal statement.
Information systems are typically utilized by a variety of users. It's wonderful that "Buyer can pay for the order" exists, but what about "Admin may fulfill the order" and "Executive may see a number of orders?"
There are a variety of non-functional criteria that are also known as quality characteristics. Karl Wiegers notes that improving one non-functional requirement may negatively influence another.
For example, we have a checkout store that needs additional security. At the same time, security harms performance. But it's evident that the checkout should be as quick as possible. Conflicts like these might result in an exponential increase in the development effort, so discussing extra quality characteristics before project estimation is a good idea.
Although project progress is always moving from A to B, new sorts of requirements emerge along the way. For example, if we introduce a new system while keeping the customer and order data from an older system. In this situation, we must migrate the data from third-party services to implement the project successfully. This is a transition requirement. Another example is instructing staff on how to utilize the system. Unfortunately, these sorts of requirements are almost always underestimated.
It is essential to understand that an estimate is not a commitment or quote; it's simply an analytical process. Project estimation should only be used to assist with decisions, not as a tender bid. Here are some tips for making business estimates more useful:
Around 70% of projects do not fulfill expectations.
It's impossible to remove the factors involved in underestimating, such as scope changes, optimism, and unforeseen technological difficulties.
A good guess is honest, precise enough, and accurate enough to enable you to decide whether or not to begin a current project.
Counting and calculating are a lot more accurate than guessing.
If the upper bound of the anticipated range is not at least twice as big as the lower, it's a bad estimate.
Exact requirements will vary, so defining them in detail before starting a new project does not aid in estimating. User stories are sufficient.
Common causes of significant underestimation include loss of goals, users, non-functional, and transition requirements.
I hope this post is more than a piece of content but also serves as a manual for executives who must assess them.
The surface temperature of the Sun - 10,000°F/ 6,000°C
Latitude of Shanghai - 31 degrees North
The year of Alexander the Great's birth - 356 BC
The total value of U.S. currency in circulation in 2004 - $719.9 billion
Area of the Asian continent - 17,139,000 square miles, 44,390,000 square kilometers
The total volume of the Great Lakes - 5,500 cubic miles, 23,000 cubic kilometers, 2.4 x 10^22 cubic feet, 6.8 x 10^20 cubic meters, 1.8 x 10^23 U.S. gallons 6.8 x 10^23 liters
Worldwide box office receipts for the movie Titanic - $1.835 billion
Number of book titles published in the U.S. since 1776 - 22 million
The total length of the coastline of the Pacific Ocean - 84,300 miles, 135,663 kilometers
Heaviest blue whale ever recorded - 380,000 pounds 190 English tons 170,000 kilograms 170 metric tons
You might find it intriguing to read this article on the potential harm of using the software development life cycle (SDLC) methodology.
VT Labs does agile development for Shopify stores, to move faster than competitors. You get store improvements every week and can change priorities anytime.
