Is Optimising for Efficiency a Fool’s Game?

Recently my Twitter feed and the blogs I follow have featured an unusually large amount of articles on artificial intelligence. According to the author Sam Harris, the often-stated goal of AI research is to create human-level intelligence. However, he points out that it is a false goal. The computers we use today already possess superhuman capabilities of e.g. computation and storage of information, and that "any future artificial general intelligence (AGI) will exceed human performance on every task for which it is considered a source of ‘intelligence’ in the first place.”

This is a rather powerful vision. Considering how quickly the ability of computer programs to understand natural language has progressed (see, for example, IBM’s Watson), and how digital assistants such as Siri, Google Now, and Microsoft’s Cortana already possess believable communication capabilities, a future depicted in the movie Her, where we communicate verbally with an actual AI, suddenly seems a lot closer.

Our entire way of life has changed radically since the industrial revolution...

It would be easy to dismiss this as overoptimistic technophilia, but the fact is that the history of technological advancement progresses in a non-linear fashion. Our entire way of life has changed radically since the industrial revolution some 250 years ago, but within that time frame you can also identify many smaller periods of significant advancements. A pattern emerges. The time frames have consistently become shorter.

Consider how technology has affected our lives in the past five years, then past 10, 20, 50, or 100 years. In order to predict the next 10 years we cannot look at the past 10 and draw a straight line. Every consequent year in the future will see more rapid advancement than the year that came before it.

Are there limits to this? Possibly, but there are also hints of technologies – such as AI – that may result in larger leaps that impact practically everything that comes after. Just consider all the aspects of everyday life that have become affected by the internet in two short decades, or the fact that the horse remained state-of-the-art in human mobility for thousands of years before the invention of the automobile.

The problem with efficiency, however, is that it requires a relatively stable environment.

The challenge this presents to organisations is one rooted in their past. If you look at the history of management practice [1], you can see it has largely been a race towards efficiency, although the methods and language have changed over the years. The problem with efficiency, however, is that it requires a relatively stable environment.

When you try to optimise a system, it means reducing variance in the operating parameters of that system. Where an established organisation has found an efficient way to produce certain goods or services for its customers, it has also become restricted by the way those goods and services are being produced. The more specialised and efficient a production line is, the more difficult and expensive it tends to be to reconfigure it – beyond the accepted level of variance that is. Each McDonald’s restaurant needs to follow McDonald’s operations manual, and changing it is not a trifle matter.

This is what allows start-ups to challenge industry giants. As the business environment changes, the optimised system of an established organisation may no longer be very well optimised at all. A new performance peak has emerged, and an agile, quick-learning start-up is often in a better position to discover that peak. Think of Kodak, which actually pioneered digital sensors for cameras but was so caught up in its own organisation, trapped in the beast of its own design, that its attempts at creating new kind of organisational alignment, better suited to a digital world, never succeeded [2].

...trying to optimise for efficiency is becoming a fool’s game.

Efficiency, while it does work in a stable environment, comes with reduced ability to adapt to changes – be they internally induced or originating from outside the organisation. As business environments are changing faster than ever, trying to optimise for efficiency is becoming a fool’s game. The time, energy, and effort that optimisation takes has diminishing returns on investment. And in fast-moving industries many optimisation efforts may already arrive too late when they get done.

However, I do believe that organisations can be designed in such manner that they can capture the best of both worlds: the key idea is to recognise which areas are changing faster than others, requiring more adaptive capability, and which ones are relatively stable and can therefore benefit from traditional performance improvement measures. For example, tax accounting tends to be an area that is highly structured, and where optimisation and eliminating variance is something to be desired. On the opposite there are areas that constantly need to reinvent themselves, such as product and service innovation.

In many of the in-between areas there are usually specific things that might benefit from more rigid structures and processes, but also multiple aspects where overenthusiastic top-down control results in nothing but loss of adaptive capability and ability for self-renewal.

Another rule of thumb to keep in mind is Pareto’s 20/80 principle, or that 20% of effort tends to account for 80% of the results, and each subsequent percentage of performance improvement becomes more and more costly to achieve. The principle can also be inverted: 80% of the adaptive capability of a system is lost in order to achieve that final 20% increase in performance. Extrapolated further it can also mean that 64% of the total effort (i.e. 80% of 80%) goes to achieving the final 4% (20% of 20%) increase in efficiency, and vice versa in the lost adaptive capability.

When seen from this perspective, the role of managers and leaders changes. They become architects and designers who build structures and conditions that steer their organisations in the right direction, but are loose and flexible enough to allow for bottom-up, self-organised, goal-oriented activity to emerge. And in self-organisation lies the key for achieving adaptability and innovativeness, while staying true to the organisation’s business goals.

For a deeper dive into the topic, I would suggest the books Competing on the Edge by Shona L. Brown and Antifragile by Nassim Nicholas Taleb.


  1. Kiechel, Walter III (2012). The Management Century. Harvard Business Review, November 2012, 63-75.
  2. Lucas Jr, Henry C., & Goh, Jie Mein (2009). Disruptive technology: How Kodak missed the digital photography revolution. Journal of Strategic Information Systems, Vol. 18, 46-55.