Monday, August 23, 2010

Good software starts with good people - not cheap people

In well over a decade of working for software consultancies, I have observed a recurring problem that I find personally frustrating.

Total cost of software development is a very difficult figure to calculate - and even more difficult to compare across projects.  Because of this difficulty, some corporates attempt to use developer day rates as their measure of spending effectiveness.  This approach tends to mean that they hire only the cheapest developers available.  They then turn to "consultancies" to provide additional "expertise" when quality plummets and costs begin to spiral out of control.  The expectation is that the consultancy will provide better developers than they are willing to hire themselves.  The client often seems to assume that, because they are paying more for a consultant than for in-house staff, the consultant will be more skilled.  

Unfortunately, this approach is driven by a fundamental lack of understanding of the software development market.  In reality, the inflated price of a "consultant" is usually driven entirely by the margin that a consultancy must charge in order to themselves make a profit.  This margin is often quite large, as a consultancy must cover their overheads while they are short on work - not just while their staff are gainfully employed.

A consultancy is driven by entirely different incentives than its clients.  While their clients expect (in some fashion) to profit off the software they write, a consultancy expects to make money off the writing of the software.  Unless they charge a margin, they will make a loss.

Although hiring better developers can lead to a better quality product and lower development costs, (and my own experience has shown this to be the case, over and over,) these factors are not of direct relevance to a consultancy.  When they are done, they will move on to the next project without ever having to actually use the software.  If the software achieves it return on investment goals, they will generally not see a dime.  Providing better quality (and therefore more expensive) staff is only profitable to a consultancy if they can charge a higher margin - and that is on top of their base cost!  Given the propensity for client organisations to seek lower day rates, it is sometimes difficult to keep a senior consultant charged out at the margin necessary for profitability.  One thing is certain - those clients willing to pay top rates are the ones that get first call on the top talent.

Because they derive no direct benefit from better quality software and/or lower delivery costs, most consultancies (despite what they might prefer) will attempt to address quality and cost only to the extent demanded by the market.  (i.e. their clients.)  The same market that sees most of their clients more driven by day rate than by total cost of development, also forces them to adapt their own hiring practices accordingly.  (In fact, I have personally worked for a consultancy that tried to adopt to the opposite approach and only hire the best possible developers.  Unfortunately, they tended to find that their clients did not want to pay the inflated day rate that the best developers required.)  Most software consultancies are forced to just "build the pyramid".  i.e. Hire a few senior developers and supplement (or swamp) them with a multitude of minimum-rate staff.

So, those very organisations that stand to profit the most from hiring top-quality development staff, tend to instead pour their money into paying consultancy margins in order to sort out the problems that they create by cutting down on day rates in the first place.  At best, they will sometimes be lucky enough to find a consultant who is actually worth the margin they are paying.  At worst, they will end up with the cheapest (although still expensive, compared to their in-house staff) consultants they can find and simply "prove" to themselves that more expensive developers are not really worth the extra money.  Regardless, they then get rid of the consultancy (in order to cut costs) as soon as the immediate problem is solved - and the cycle begins again.

Almost laughably, those organisations sometimes then try to address the inadequacies of their minimum-rate development staff by dictating in detail how they should go about their jobs - as if they had a clue themselves!

To those organisations that I am describing, I would like to note, for the record, that I have personally (on too many occasions,) replaced teams of minimum-rate developers with developers whose day rates are 3 times that of the incumbents and still realised overall cost savings of greater than 50%.  (Note that before-and-after comparisons of total development cost are much easier to draw than cross-project comparisons.)  While this might keep me employed, (or run off my feet, more often than not,) that employment is (in light of the larger picture I have attempted to paint here,) gainful only to myself and hardly fulfilling. Perhaps it is time to consider another approach?

 The only path to good software is through good software developers.

4 comments:

DreamCatcher said...

Point well written.Unfortunately cheap is the way most organizations are going nowadays. Just a matter of time before we realize the folly.

Kim said...

Fuly agree!
Now the challenge; how to turn around the mindset.... and provide a proof method to convince the people that need to spend the cash?

Milan Gupta said...

Aaron, the challenge is how to scale the model you propose across the enterprise. If you can figure that out, you will be famous.

ADK said...

In response to Milan, I agree that scaling this approach is challenging. However, I would make a few points in response:

Scaling the minimum-day-rate approach may seem "easy" in that it is easy to hire a virtually unlimited number of unskilled developers. There is an "obvious" path to follow, but does it really lead to where we want to be? I think we have all had to deal with the fallout of this approach. I don't think it really scales any better at all. A whole lot of bad code is no better than too little good code. (Substantially worse, I would argue.)

In many organisations, it is actually easier to hire top talent, than it is to get permission to hire them. Sure, there will be a limit to how many top people are on the market at any time, but it would be nice to have the opportunity to explore those limits, once in a while!

We should remember that talent and experience are not the same thing. Talented juniors are valuable team members - and they are essential, if we are to have another generation of senior talent.

The challenge of software development at enterprise scale tends to lie more in governance than in project execution. I have run enough teams in my time to be confident of repeatably producing better (and cheaper) results with small, skilled teams, that will large, unskilled teams. The state of the art is such that, at project scale, skilled developers (and testers, analysts, PM's, etc) can reliably produce good software.

It is the governance and co-ordination of large numbers of teams that is difficult. With skilled teams the challenge moves from "how do we deliver anything?" to "how do we make sure that we deliver the right thing?" Not such a bad problem to have! ;-)

However, isn't the governance of even larger numbers of less skilled people even harder? Perhaps what we need is more skilled governance?

I don't want to downplay the challenges of governing an enterprise scale software house. I doubt I could do it myself. I suspect that enterprise governance technique and philosophy simply hasn't evolved to the same degree as have modern software project execution approaches.

IMHO, John Seddon's "Systems Thinking", which draws heavily on "Lean" philosophy and the model established with the Toyota Production System, sets an example of what good looks like here. Widespread acceptance and understanding of this approach seems to be missing. (By the way - John Seddon is now reasonably famous!)

Milan, I'll pose you a question here: is it really any easier to "scale" with unskilled hoards than with cheaper, better developers? What are the real challenges? Aside from convincing the bean counters, are the challenges of governance not simply added to by poor quality output from unskilled teams?