“We Can’t Afford to Spend Time On Automation”

What does it cost you to test your product before every release?

It’s common, especially in the early going of a company, to do all release testing manually. It usually takes the form of something like “Let’s get everyone in the company to test the app before we release it.” Sometimes there’s a formal test plan, but lots of times there’s a lot of hoping and very little structure.

If you’re the business owner, this seems like a reasonable strategy. After all, you don’t want to hire a formal QA team yet because that would be too expensive. And, when your developers tell you it’s going to add 20% to automate testing, that seems too expensive as well.

Very few companies actually do the math on what this methodology costs you because the costs aren’t real. You’re already paying the people at your company, so to have them spend 4-5 hours a piece on testing doesn’t affect the bottom line. Or so goes the myth.

The True Cost of Manual Testing

Here’s a simple calculation to show how much this practice is costing your company.

Let’s assume you’ve got five employees, all who make an average of $60,000 per year, for an annual run rate of $300,000. That’s about $150 per hour plus or minus, not including benefits, taxes, etc. If each employee spends five hours testing, that’s $3,750 per release. Most companies do at least six releases a year in the early going (some do far more), so that’s $22,500 a year just to do the work of manually testing.

As the application grows in size and complexity (which it always does), the test cycle gets longer. You’ll also start to miss things in testing because humans are really not that great at doing this kind of work. It’s boring, and people have their real jobs to do. The test cycles will lengthen without you noticing. It’s like boiling the frog. All of a sudden everyone’s spending 10-15 hours testing every release, and your release quality is getting worse.

To compensate, you add more manual testing, which means you don’t release as often, which means your customers aren’t getting the features they want as quickly as they want, which means they start looking for other products.

What’s the Point?

If you’re stuck in primarily manual testing of your product, have you actually sat down and done the calculations to see what it’s costing you in hard labor costs, not to mention all of the more soft problems that hide themselves in additional support requests, more bugs, and less user satisfaction?

As with most hidden costs, the number is usually far larger than what you think.

In today’s environment, letting humans do what a computer can do for probably 1/10,000th of the cost is foolish.

Just Remember

The example I gave above is actually very conservative. I’ve seen companies of 20 people or more spend the better part of a month testing a release. The costs in both near term and long term effective use of capital are staggering at any sort of scale.

If you’re just getting started, don’t fall into the myth that manual testing is cheaper. It never, ever is.

If you’re further along, and the above scenario sounds familiar, the time to start working on this problem is now.

Your Assignment

Set up a spreadsheet with each of your team’s effective hourly rate if they’re involved in testing releases. The next time you do a release, have everyone record exactly how many hours they spend doing their testing.

If you’re like every other company whose gone through this exercise, the total is going to be a bit of a shock.

The path forward, once you understand how it looks for you, is to introduce automation as quickly as possible. Here’s some tips to get you started, and as always, leave me a note in the comments if you’d like to know more about your specific situation.