Premature Optimization: Root of All Evil or Rationalization?

Special contributor to Scout APM  on July 06, 2021

Premature optimization refers to the process of making a piece of code more efficient when it is too soon to guarantee an optimization. To make it more clear, when prematurely optimizing, you usually spend a lot of time and resources working on parts of your code that will have a negligible impact on the outcome rather than on those things that will heavily affect the final results. Such measures almost always backfire since you miss out on perfecting the metrics that truly affect your business.

undefined
Source: Structured Programming with go to Statements


Additionally, there are high chances that you will create new problems as you try to optimize things way before there is enough data to make an informed decision. For instance, you might make changes that appear rational while your app is in its initial stages but make your codebase inflexible as you reach the final stages of the production. Therefore, understanding premature optimization and avoiding it is among the essential tasks for any software development team.

In this article, you will learn about the term in detail and understand what it is and what it is not at the same time. You will learn ways to avoid premature optimization and how to not skip out necessary optimization in the name of early efforts.

Feel free to use these links to navigate the piece:

What Do We Mean by “Premature Optimization”?

Somewhere in your programming career, you must have come across the famous phrase “premature optimization is the root of all evil.” This quote has grown from an idea to a best practice among young software developers. The term premature optimization here refers to optimizing an application in a way that doesn’t add any value or at a time when it’s too soon to tell whether the optimization will be worthwhile and produce the intended benefits. This brings a variety of problems with itself, such as waste of resources and time. The quote aims to instill among developers the message that optimization has its own appropriate time, and you should not do it sooner or later.

However, the meaning of this quote has changed quite abruptly in the modern software industry. Software engineers have often even shortened this idea to “optimization is the root of all evil.” It is unfortunate to see this twist, but many misbeliefs and myths have ultimately led to it. The ignorance towards understanding the real intent behind the quote leads to more significant problems than premature optimization itself.

Origins of the Saying “Premature optimization is the root of all evil.” 

The saying originated from the book The Art of Computer Programming by Donald Knuth. Here is the complete quote from the book:

“The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming."

Sir Tony Hoare initially gave this quote and has been popularized by Donald Knuth through his book. The quote was from around the 1960s, when computer hardware was just beginning to gaze at the coming dawn of possibilities. 

Around this time, CPU processing was costly, and punch cards were a popular storage alternative. Most computer applications could strictly use only the resources they needed; therefore, the quote made much more sense. But in a world like today, there’s enough CPU and memory to use all the resources you want, and not just need. However, today’s necessity of optimization aims to cut down unnecessary costs involved in running non-optimal systems.

Hoare’s saying has been twisted and manipulated by many developers to escape the process of optimizing their code. People tend to postpone the optimization step until the very end of the software development lifecycle. When they reach the end of the development lifecycle, the optimization phase misses out due to economic or time constraints. In reality, Hoare never said that optimizing in the early stages of the software development process is evil. He meant that doing it during the foundational phase of your application, when the codebase is fluid, is evil.

When “Premature Optimization” is Taken Out of Context

While the phrase premature optimization is the root of all evil intends to direct programmers away from wasting time in insignificant attempts at optimization, it has taken a different path lately. More often than not, people perceive the quote as “optimization is the root of all evil” and hence skip out on optimizing an application altogether. 

Several other common misbeliefs about optimization exist in the software development community. Some of them attempt to “counter” the effects of premature optimization, but all that they do is optimize at a point when it’s not even possible to make significant improvements. Let’s take a look at some of such myths:

Myth #1: Optimization is only about performance

Many software teams believe the process of optimization factors in when an application is ready to be deployed. Therefore they tend to ignore the importance of optimized design earlier and try to improve the performance of an application by making local changes in the code.

Myth #2: Optimization is easier at the end

Software developers often misquote the 80/20 rule while deciding on the performance optimization of an application. They believe that since only 20% of the code takes 80% of the execution time, it will be easier to fix it towards the end. In reality, this 20% gets scattered throughout the codebase and is often impossible to modify entirely. If your code does not follow optimal design patterns, it becomes challenging to change it for significant performance improvements later.

Myth #3: By the time my code is published, machines will have improved

Many software engineers believe that by the time they ship their code to end-users, the standard hardware performance will improve so much as to cover any design sloppiness on their end. This might seem relevant for a time like the 1990s, but today’s hardware growth does not match the then rate. Ideally, it is never wise to depend on any external factors for enhancing your app’s performance.

Myth #4: It is difficult to predict the execution time of an app’s segments

However astonishing it may sound, some software engineers unaware of the modern advances in observability believe that it is nearly impossible to predict the execution time of their code’s segments. Due to this, they skip out even those sections that unambiguously lack in performance. In reality, many monitoring tools like Scout help you predict where your app will lose out on performance before releasing it into production.

Myth #5: Optimization can waste your time

Many developers believe that you could put the time you spend optimizing a piece of code to better use by writing more code. This can help you reduce development time and hit the market faster. However, they forget that you will have to roll back and improve anyway if you hit the market with a poor-performing product. Moreover, designing a fast app is always easier than enhancing a slow app’s speed. To sum up, while optimization might appear to take up precious development time, it will save you from a lot of even more precious maintenance downtime.

Myth #6: Micro-optimization never helps

More often than not, engineers argue that there is no point in optimizing a piece of code if you can not suggest a better way of doing the task altogether. They press on finding better algorithms instead of optimizing any algorithm’s implementation. They also advocate that this is what the first part of the quote means. However, we shouldn’t rule out micro-optimization entirely. If there are no avenues for making significant improvements, you can always try to work on the more minor but worthwhile enhancements.

Myth #7: You can always choose a better algorithm later

One of the most prominent myths that most software engineers believe in is that you can always come back and change your code. While this is true for more minor, contextual changes, this does not apply to the bigger picture. It is near to impossible to update the design of an application’s code without rewriting it altogether. Therefore, it is always advisable to prepare optimal plans and choose the best algorithms before writing the code.

Why is Premature Optimization Inherently Bad?

Premature Optimization is inherently inefficient. Optimizing software in a way that isn’t clear to bring about improvements or that can’t be measured stacks the odds against you. Here are some reasons why it is never a good idea to jump the gun on optimization.

Why Premature Optimization Leads to Inefficiency

A consequence of premature optimization is optimizing in the wrong places, often overlooked when talking about the famous phrase. In any application, some sections account for 2% of total execution time, and some account for 20%. It is evident that if you optimize the former, you can at most get an improvement of 2%. But if you manage to optimize even half of the latter, you can gain around 10% overall improvement, which is way better. This is what we mean when we say “finding the right places.” 

If you wear yourself out perfecting the 2% segment, you will have worked so much to achieve so little. As you can imagine, this will be inefficient and will stop you from working on things that need more attention, like the 20% segment.

Premature Optimization Wastes More Than It Saves

In almost all cases, the prematurely optimized code will cause problems later in the development process. From inflexibility towards improvements to a requirement of a complete rewrite, premature optimization will only cause you to spend more time and resources. Instead of putting efforts into optimizing earlier and then again putting efforts into reversing the damage, you should consider waiting for the right time before you set down to optimize your application.

Optimizing Prematurely Lowers Team Morale

While you think you are saving your team some trouble down the line by optimizing earlier, you are only stacking up more work on their plates. Each line of code that requires rewriting takes up more effort than the ones written from scratch. And if somebody else has to clean up for your mess, it will only create collaboration problems within the team.

You Are More Prone To Mistakes When You Optimize Early

Premature optimization forces you to work with the information you have at hand, and in most cases, it might not be the complete picture. It is essential to understand that optimization leaves software open to more errors and issues since there is an attempt to balance performance and purpose. But when this optimization is carried out earlier than usual, there are high chances that you might introduce unsolvable errors, which might force you to redo your application from the beginning.

Premature Optimization Unnecessarily Hinders Progress

Optimization is a time-intensive operation. You need to rethink your approaches, learn new ones, and try out a couple before deciding on the best one for your use case. Optimization implemented towards the end of the software development process doesn’t affect its release. But when you try to do it at the beginning of the development process, you introduce unnecessary delays in your roadmap and often wander off into unplanned paths. This can cause a severe diversion from the planned roadmap and can cost you weeks to recover.

Don’t Let Premature Optimization Be an Excuse for Not Making Necessary Optimizations

One of the biggest mistakes you can make is skipping out on necessary optimizations by flagging them as premature. When Hoare said, “premature optimization is the root of all evil,” he did not mean “optimization is the root of all evil”. As a software developer, you must accept that optimization is a mandatory part of the software development process, and you can not dismiss it simply by flagging it evil. Like testing, quality assurance, and maintenance, optimizing needs to be given its share of importance.

Optimization is complicated, no doubt. Along with choosing what to optimize first, it’s also challenging to find the right time and place for it. But avoiding it altogether will not fix any problems. The only way to get better at optimizing is to practice it more often. Testing is a complex process too, but we hardly see developers skipping on testing because of its complexity. It is high time that we start treating optimization similarly.

What to Focus on Instead of Optimizing Around Minutia

It is important to note that Hoare’s complete quote is, “We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.”. The developer community often ignores the first part of the quote, but that is where the solution to avoiding premature optimization lies. Minor efficiency improvements might not be worth the effort during the early stages of the development process. 

However, you can not ignore minor improvements entirely. They may not appear as a significant upgrade to your metrics, but they can still bring considerable improvement if done at the right time. The key here is to understand the difference between an optimization attempt at the right time and otherwise. This comes through experience, as Charles Cook’s quote goes:

A good software developer will do this automatically, having developed a feel for where performance issues will cause problems. An inexperienced developer will not bother, misguidedly believing that a bit of fine-tuning at a later stage will fix any problems.

Here are some measures that you can implement to find out the right way to optimize your application.

Find What Needs to Be Optimized

The first and foremost step in the process of optimizing efficiently is to find what needs optimization. There can be multiple areas of code that might appear to be causing performance drops, but not every one of them will be worth pursuing. You need to figure out a way to identify the feasible regions and get the most out of the time you invest in optimization.

The obvious question that pops up now is—how does an amateur programmer know when optimization is feasible? The answer is to profile as much code as you can. Measuring the time that the control spends in each segment of your code will help you understand which parts take higher than the usual amount of time. You can try profiling similar code segments to compare their performances and determine which one is easier on your hardware resources.

APM tools are built just for this purpose. You can use Scout to log and analyze the performance of your application. You can utilize Scout’s extensive instrumentation and logging features to extract performance data from each line of your code. With abilities like memory bloat detection and tying bottlenecks to the line of code causing them, you can quickly identify the sources of most performance issues in your application.

Also, as an amateur programmer, you should be ready to make mistakes and learn from them. There are high chances that you might end up optimizing the wrong piece of code in some cases. But all that this will do is add a minor amount of performance boost to your application. You will perhaps lose some precious time working on the wrong task, but this experience will likely help you better identify higher priority code segments.

Take User Feedback Into Account

User feedback is a tool that holds much more power than the average developer believes it does. You can do a lot of market research, develop the best strategy for acquiring users, and give it your best shot in the first attempt, but if your users find something off with your app and you are unaware of it, you are digging your application’s grave.

User feedback helps you to shape the roadmap of your application’s evolution. Strategy and market often don’t agree with each other, and what seems obvious on paper can sometimes utterly fail in the real market. User feedback protects you from this disparity by helping you consider the market’s demands while designing your strategies.

Application performance, scaling capabilities, unique feature set; all these factors in when your application hits the air and starts getting some traction. But when you are yet to roll it out, you need to worry about more important things like – designing UI mockups, running user surveys, prototyping and rolling out a lean, core-features-only version of your app, prototype acceptance rates, etc. All of this is nearly impossible without utilizing product feedback.

You can always spend a lot of time in something menial, such as perfecting your deployment pipeline, but what needs your attention in the building stages of an application’s life is user feedback. So, focus first on building an application that people will love to use; optimization and improvement can always follow.

Prepare a Robust Strategy For Optimization

Optimization is a somewhat complex process. There are endless possibilities both in terms of products and techniques available to optimize your application. Therefore, it is all the more critical to decide on a fool-proof plan and stick to it. 

Implementing any process requires a sound plan. Optimization is no exception. Finding the proper strategy and time is crucial to optimizing your application and the process of optimization itself.

Instead of wasting your time on things like rewriting a part of code that does not strictly follow a set standard or looking for random, negligible improvements in performance, you should focus your attention on the bigger picture, which includes –

To carry out a compelling round of optimization, you need to prepare a plan and put in only the most essential things. Without a plan, you will only be making minor, less worthwhile improvements to the app’s performance while unknowingly opening it up to new errors and issues.

Plan Your Projects Well

Since we are talking about planning, let’s take a look at another phase of planning which can help you skip out on performance issues altogether. A necessary adjustment that software teams need to make is prioritizing performance in the designing phase of any software project. 

Products that don’t keep performance in mind tend to go down paths from where it is impossible to return to a high-performance state. More often than not, the only way to improve from such situations is to refactor the app from its roots and redesign it all the way. As I mentioned earlier, too many software teams read premature optimization as just optimization in Hoare’s quote and shamelessly ignore the need for optimization in any part of the software development process.

Hoare’s saying applies to the optimization phase only, and it nowhere states that you should not consider performance in your application design. Also, optimizing your app while in the design phase does not count as premature; that term is meant specifically for the unwarranted on-the-fly optimizations made in the development phase of the software process. So, always remember to imbibe good performance in the design of your app because while premature optimization creates difficult-to-modify code, lousy designs create code that requires rewriting entirely.

[For Developers] Know Your Tools and Languages Well

You can make a great plan but still fail to execute it properly if you aren’t entirely familiar with your arsenal. For instance, without knowing the mechanics of garbage collection in your language or how often a particular framework rebuilds its component tree, you can’t expect yourself to make informed decisions in an ocean of optimization possibilities.

It is easy and convenient to assume that each line of code roughly takes one unit of time to execute, thanks to the modern Data Structures and Algorithms courses. However, in the real world, there are high chances that these lines of code are incredibly abstracted – that with one line of code in your application, you could be running an entire miniature application internally in the form of a library. Each of these single lines of code can take up highly varying amounts of time to execute, and with more and more third-party plug-ins in the market, you never know which line of code is a ticking time bomb to your happily running application.

Therefore, you must know well your programming languages, frameworks, libraries, third-party add-ons, and everything that goes into your application. You must have at least a basic sense of which tools are easy on the hardware and which of them are capable of frying CPUs. Rico Mariani has a perfect quote for this situation—“Never give up on your performance accidentally”. It is easy to give up performance accidentally; all you need is a pinch of ignorance while writing code. But, this ignorance can cost you hundreds or thousands of dollars if not taken care of at the right time. So, always know your tools well, and be vigilant while using them.

Be Efficient, Not Lazy

Quoting Rico Mariani again, “Never give up your performance accidentally”, it is essential to focus on the term accidentally. You give up performance accidentally when you don’t understand your constructs well or are too lazy to build one for yourself before coding. To curb this, you need to study how the system translates the code written in your high-level language. You can then use this understanding to improve the quality of design at all steps of software development by comparing the costs associated with multiple available approaches to achieve the desired results.

In many cases, you might want to give up on performance intentionally. This can be due to various reasons, such as improving maintainability or code readability, and does not go against Rico’s statement since it is not accidental. You are making an informed decision aimed at doing your application good in some way.

On the whole, Hoare’s statement has been responsible for driving a fundamental change in the way modern software development goes around. Some have, however, found ways to misinterpret the message to use it as an excuse to skip out on crucial optimization. It is essential to reverse this since it can wreak unnecessary havoc on your infrastructure and bills.

Reversing this decline of optimization in the industry is not an easy job. However, you can begin by acknowledging the fact that the prevalent attitude is wrong. Once you know how you are missing out on crucial performance upgrades due to ignorance, you will automatically want to fix it by looking out for performance boosts. This is where the quote comes in—you want to look at places that are feasible for optimization. You don’t want to waste time micro-optimizing when more significant performance boosts are available in other areas. You also don’t want to miss optimization because you can not find a performance gain feasible enough. Striking the perfect balance between the two is key to optimizing your application efficiently and effectively.

If you are looking for a tool that helps you identify performance bottlenecks quickly, make sure to check out Scout APM. Scout is cost-efficient, offers excellent customer service, and provides the best-in-class developer experience in APMs. The interface is simple, easy to understand, and the initial setup process is free of any hiccups. You can get started with our APM services with a 14-day free trial without needing a credit card.