Building a Digital Assembly Line with Continuous Delivery
Imagine if you walked into the Tesla factory and saw all of the welding being done by humans. You’d probably wonder what in the world was going on, and you’d probably not buy a car from them because it would be too expensive and more than likely a lot less reliable. Yes, there are cars that are built by hand, but those cars have names like Lamborghini on them and cost more than $250,000. They also make only a few cars a year relative to the rest of the auto industry.
There’s nothing wrong with making a car by hand, but you can’t make very many of them and they naturally cost a whole lot more.
The same is true of assembling and distributing your software product. The more you are doing manually, the more it costs and the less reliable it becomes over time.
Continuous delivery takes all of the steps required to release your product to the general public and reduces them to a process that computers manage on your behalf, allowing for faster, more reliable production of your product.
Get Rid of the Checklist
Lots of product companies are still saddled with the old, manual method of software delivery. They create builds manually, they test manually, and they deploy manually.
Each release has a long checklist that someone manages (in a best case scenario) with a checklist. Many times there’s no checklist or other documentation either, which means (naturally) that things occasionally get missed. We humans are not built for repetitive tasks.
The checklist that someone’s using to release your product manually is the start of your continuous delivery process.
Death by a Thousand Cuts
Low-functioning product teams are characterized by highly manual processes. One of the first things I do when I work with companies that are having product quality issues is to have them write down their testing and delivery processes. I force them to write every, single step down, no matter how small it seems. It’s not uncommon to see things like:
- “Manually increase the version number”
- “Run database update scripts”
- “Copy the build files to all servers”
Imagine a 15-20 step checklist full of items like the above. It would take 5-10 minutes at a minimum each time a delivery is done to a machine, multiplied by the number of machines and number of releases. It’s not uncommon for companies to spend $15-20,000 a year just deploying their releases, and this is at very small scale.
It’s a terrible use of smart people’s time, and most smart people will find somewhere else to work at some point. It is mind-numbing work.
Getting Unstuck
If your team is stuck in a manual delivery process, it may take a bit to change the mindset of your team. Going from manual to automated delivery will likely never seem to make sense, particularly if you’ve got an involved process for deploying your product. The initial cost of automation will seem prohibitively high.
If you’re struggling with where to start, the best thing to do is to have your team make a list of all the steps required to push your product to your users. The list needs to contain every single step, such that someone with no experience running your deployments could step in and use those instructions with no further details. You can test this by using the checklist on a test or non-production environment.
Once you’ve got a complete, working checklist you can then start picking items off the list to automate. Starting with simple things like copying the build files to all servers creates simple, small wins that start to build momentum toward full automation. You’ll be amazed at how quickly everyone on the team gets the fever for full automation.
Just Remember
Continuous delivery will cost you some time and money at the start of the your project, but it will cost far less than the time and money you spend to do it manually down the road, and the longer you wait the more painful it becomes. Manual work-arounds will slip in over time that create more room for errors and omissions.
Thinking of your product as the end result of an assembly line creates the proper mindset for managing all of steps of production. Computers are great at repetitive tasks and cost orders of magnitude less than humans doing the same work.
Your Assignment
Meet with your team and have them walk you through the current deployment process for your product. If you see any manual steps, immediately have your team write down the entire process like I described above, and right beside each step have them write down how much time it takes. Multiply that time by the hourly wage of each of the people involved, then multiply that by each release to any machine.
I’ve never had anyone go through this process who wasn’t shocked by the result. It’s a big number, and that’s not even factoring in the opportunity cost of the people doing these menial tasks or the quality issues you’re going to have over time.
If you’re already using continuous delivery for your entire build pipeline, buy yourself a Snickers bar from the snack machine and pat yourself on the back.