Estimating software projects using Fermi questions

How many times do you need to estimate a software project with hardly any details? In the work of a software engineer it happens frequently and most of the time the details about the projects are not yet available (and probably will never be!). 

To achieve an estimate there are two kind of approaches:

1. Non-algorithmic approach: usually is done by experts who use their experience with similar projects to estimate. This method is typically used when there are hardly any details.
2. Algorithmic approach: use a defined logic with parameters for the estimation. 

Here you can find a lot of details about these two approaches. In this article I want to examine the way to achieve an estimation with the way that seems to be prevalent: the so-called “expert judgment technique” with a non-algorithmic approach.

Fermi questions

A “Fermi question” is a question which seeks a fast estimate of quantity impossible-to-measure directly and in situations where very little information is known. In fact Enrico Fermi was famous for being able to make good estimates of these kinds of problems.

Example: How many grains of rice are there in a 10kg bag?

It seems almost impossible to answer these kinds of questions. If you have to estimate a software project you could have the same feeling because the software project could have too many unknowns. 

In the following I would like to describe a possible way to approach a “Fermi question” and what are relations with the software engineering world.

Breaking into sub-problems

Do you know the saying “There is only one way to eat an elephant: a bite at a time”?

To solve a Fermi question, the first thing could be to try to break the problem into sub-problems. Each problem can be analyzed separately in order to reduce the complexity of the main problem. 

You can do the same thing if you need to estimate a software project. A method for doing this is to split the project into “atomic” features and split it again into tasks. For a particular feature an example of tasks could be:

1. Produce a technical analysis
2. Create data structure (∼ 3 tables)
3. Create UI  (∼ 3 pages)
4. Implement business logic
5. Create an API to expose data
6. Automatic test
7. …

Estimate the sub-problems

After splitting the problem into tasks, the estimate can proceed. The simplest way is to have one or more experienced persons to define the effort needed for each task and sum it up! 

This approach is really simple but it’s definitely not a scientific method. In science and statistics it is not enough to identify a single value for your measure (in our case the estimate!) but it is always necessary to also calculate a standard deviation. In other words, your estimate should be something like this:

155 ± 15 (points/days or other)

This allows you to define not only an estimate as a single number but also a standard deviation (in this case 15) that indicates how accurate your estimate is.

There are different methods that you can use to achieve this. One of the most used is the Program Evaluation and Review Technique (PERT) that consist in defining three types of estimates per each task: 

1. optimistic estimate (o); 
2. most likely estimate (m);
3. pessimistic estimate (p). 

With these values is possible to calculate the expected estimate (e) and standard deviation (σ)  for each activities using the PERT’s formulas:

e=\frac{o+4m+p}6

\sigma=\frac{p-o}6

Example:

Tasks	Optimistic estimate (o)	Most likely estimate (m)	Pessimistic estimate (p)	Expected estimate (e)	Standard deviation (σ)
Task A	6	10	15	10,17	1,5
Task B	18	25	39	26,17	3,5
Task C	14	22	35	22,83	3,5

Finally we can calculate the total estimate and the total standard deviation with the follow formulas:

Total\;exp.\;estimate=\sum_{}e\simeq59,17

Total\;\sigma=\sqrt{\sum_{}\;\sigma^2}\simeq5,17

So, in this example the estimate is 59,17 ± 5,17. Curious about the math behind? Go to this link: PERT distribution.

Before ending this section, let me say: be careful about the fact that “the whole is greater than the sum of the parts”. This is the limitation of the approach to decompose the problem into sub-problems. It is always best to double check the total estimate with the experienced person in order to analyze if the “order of magnitude” is as expected compared to the projects previously analyzed.

Make accurate assumptions

To solve a “Fermi question” it is important to make plausible assumptions. In general, an assumption is a statement that we believe is true. To do the estimate we’ll need to make certain assumptions regarding the unknown elements and then estimate in accordance with those assumptions. 

The estimate should not be just a number but also a set of assumptions that you used to arrive at the estimate. In a project estimate, these assumptions should be clearly communicated to the project stakeholders. It is important to ensure a validation of your assumptions in order to understand if the estimate you have made makes sense.

Communication is the key

Estimation could be really hard! Accept the fact that you can fail – during the task realization, you can discover that the estimate was not correct. 

In that case communication could be the key: make sure that stakeholders are aware of the new problems you encountered so together you can agree on how to handle it. One of the possible scenarios is to renegotiate the project scope in order to respect the delivery dates and the budget.

Conclusions

The approach I have tried to describe to define an estimate is the following:

1. Breaking in sub-problems.
2. Estimating sub-problems and calculating the total estimate. Use a method that allows you to calculate a standard deviation.
3. Not just a number! Always write the assumptions used to define the estimate.
4. Accept the fact that human errors are inevitable and sometimes estimates can be wrong.
5. Report any problem during the project and stay open to renegotiate the project scope if necessary.

Sometimes estimating can be very difficult, but if done right, it creates an important basis for project planning. This will result in a healthy project and team who did not endure stress due to being wrong about the initial estimate of the project. Like so many things in life, estimating is learnt with experience and first of all – you need to get started.

Reference

1. A Review of Agile Software Effort Estimation Methods.
2. PERT distribution.