Why is it so hard to estimate development time?

Many junior developers I meet admit to having a major difficulty in their day-to-day work: estimating the development time of a feature or an entire project. My answer is very simple:

That's normal.

The first cause is that developers who lack experience don't have the technical background needed to make a correct time estimate.

It's not for nothing that the root of the verb "to foresee" is "to see" — to foresee something, you first need to have "seen" it at least once!

But that's not all, because even though we improve with time and experience, that doesn't mean that once we become an experienced or senior developer our estimates will be accurate 100% of the time — far from it.

The difference between the estimated time and the actual time can vary for many reasons, including:

Hidden complexity

Sometimes certain features that seem basic at first glance were proposed as an option to the client without consulting the technical team (which of course shouldn't be done), but actually hide a much greater complexity.

Let's take the example of a "simple" search bar: such a feature on a fixed list of items (or a very small list) can indeed be implemented in a few lines of code with a conditional display.

But to handle a text search across a list of complex resources whose number keeps growing over the lifetime of the application, you'll likely need to set up an external search engine like ElasticSearch, which will need to be synchronized with the database and will make the project's architecture much more complex.

Accidental complexity

Known as "Accidental Complexity" in English, this notion corresponds to the additional complexity linked to the project as it develops over time, and which must be taken into account when creating a new feature.

For example, feature X might be very simple to develop in a brand-new project, but in a project built on a complex architecture with dozens of business modules and tens of thousands of lines of code, the same feature will take much longer to integrate.

This complexity is difficult to estimate by its very nature, but also because you already need to know the project and the state of the existing codebase very well in order to assess it.

Difference in perspective

If you ask several developers to estimate the development time of the same feature, you can end up with differences ranging from double to triple, and sometimes even more.

The reasons mentioned above can be the cause, but there's also a bias in this process, because we often think that the shortest estimate will come from a senior developer who can leverage his experience.

But on the contrary, it sometimes happens that the more experienced a developer is, the longer his estimates become, because he no longer only sees the feature under the conditions where everything goes smoothly (also called the "happy path"), but he'll be able to anticipate all the special cases where care must be taken during implementation (the famous "edge cases").

So here are a few reasons that may help you understand why it is very difficult (if not impossible) to precisely estimate the time needed to complete a project as a developer, whether junior or senior.