Business vs. engineering excellence

Siding is a grave mistake in this battle whether you support the business suits or the engineers. A "compromise" approach is merely showing lack of understanding of the problem. So what's the right choice? As usual, it depends. But then, what does it depend on? When exactly is the right time to go nuts with engineering vs. business priorities? When do business interests and quality engineering go hand-in-hand?

Perfection is not the same thing as engineering excellence. Perfectionism is a mental disorder. It provides high for the brain, which craves order, control, and success. It is easy to tickle these innate desires at the cost of immersing the victim of perfectionism in illusory universe formed by selective blindness to issues outside of the island of perfection.

Perfectionism is actually a worse case than obsession with hard problem cracking, which is a similar self-satisfying behavior providing the hacker with really pleasant illusion of success. While problem cracking at least challenges human intelligence, perfectionism often involves mindless application of a small set of rules.

Business-friendly, productivity-conscious approach can get seriously weird too. It provides different type of high - fast-paced work, popularity, recognition, and a different kind of challenge. It involves a special type of problem cracking focused on business problems. You could reach stellar high pace of work and introduce many revolutionary innovations while completely missing the business objective.

Engineering excellence, even if stripped of the mindless perfectionism, doesn't really make much sense in many cases. If it kills productivity, there will be less time left for engineering later on, which will in turn result in lowered quality, which renders the original engineering effort useless. As this is a consequence of limited budget, one could slash features to prevent the near-deadline disaster, but then features are often required rather than selected.

There's one really big catch though. Excellence, rather counter-intuitively, increases productivity. Bugs discovered early are cheaper to fix. More generally, the policy of completing one component at a time results in nearly perfect temporal and spatial locality of all work processes, which has massively positive impact on productivity.

Productivity is not the same thing as pace of work. While your efforts to limit engineering investment might be (creating an illusion of) improving pace of work, you might be at the same time vastly increasing the probability that the product will be rejected, thus reducing its value and by definition reducing the productivity of any work that led to delivery of the rejected product.

Think no one will reject a product due to low quality these days? A chain is only as strong as its weakest link. Complex systems cannot be built from shit. Software is always a foundation for something else (mostly data and processes). End users of the product likely have tons of work to do. They certainly don't have time to devise workarounds for flaws in your software. They might not even have the cognitive capacity to design the kind of fault-tolerant system that could incorporate your shit without risk.

By this time, you might have noticed that I am a big fan of engineering excellence. How do I manage to get anything done? It turns out that the mindless perfectionism is a rather ineffective method to achieve excellence in anything. Reliability, performance, and even security can be bolted on top of a rather flaky prototype at very little cost.

It's actually desirable to separate quality enhancers from the main application logic, because they are essentially second order features (cross-cutting concerns, aspects, whatever you call it) that modify all other features. You don't want to inline them everywhere. You want to describe them separately from core application logic.

Is business value a quality metric similar to performance and reliability? What about business engineering, does it sound sensible? While I am not that crazy into business metrics, I do see that business value is the ultimate quality metric where all the engineering metrics get anchored. It's the source of all meaning in a software project.

Unsurprisingly, business metrics can be optimized just like any other metric, by knowledgeable engineer combining his insight with factual and statistical data. In this view, new features can actually add to product quality as they can bump up business metrics even if they hurt some engineering metrics.

The distinction between business and engineering actually follows from the old self-definition of IT business units, which separated themselves from the rest of the organization at the boundary of functional specification. Anything before spec became the business side and everything after the spec became the engineering side. Such separation is pointless in IT-heavy business and it is becoming a questionable practice in traditional IT units that are under pressure to increase their productivity.

Except where this spec membrane exists to protect productivity of the IT unit. Removing the Chinese wall surrounding the IT department might seem like a bold step of the IT guys trying to conquer new territories. But more often than not, IT units are subordinated to sales and support units and removal of the spec shield only results in sales and support executives meddling with development processes they don't understand. They will destroy the IT unit thinking they are just ensuring strong business focus.

That's why internal development and outsourcing are so boring jobs. If developers try to gain access to more challenges, they will instead find superior executives paralyzing the development process in fear that any minor refactoring could cause damage to the business.

Anyway, let's stop this undirected, frequently side-tracked brainstorming now. I hope you got some ideas. I know the answer I gave you is still "it depends". No clear-cut rules, sorry. I just wanted to make you think, at least initially, before you commit to your own obsessive approach that best fits your job.