Your automation and orchestration roadmap has some mandatory costs (Part Three)

Posted on Posted in blog

Any automation roadmap has some pre-requisites and implicit requirements. DevOps teams have to perform integration with existing systems. The deployment of a modern testing infrastructure is a must. This blog provides some background about the costs of these tasks.

A single source of truth minimizes confusion

Most automation and orchestration projects Inmanta has encountered require integration as they are not standalone projects. At least one of the company’s OSS/BSS, e.g. an inventory system, has to provide critical inputs to the automation/orchestration software. For instance a closed-loop automation will not work without an interface with the official performance management or fault management system. If not, then the automation project has probably re-invented the wheel by capturing the operational state of the resources. An organization should avoid multiple sources of truth.

Readers familiar with OSS/BSS integration activities already know how much organizations downplay their level of effort. They are usually very significant for various reasons. The unavailability of public open APIs, poorly agreed accountability when systems have multiple owners and data quality are a few examples among them. Poor motivation and lack of interest is also an elephant in the room. OSS integration is not the best way to build up a CV…

East-west APIs are probably the biggest work package at the beginning of the automation journey. How to ensure their fees don’t skyrocket over time? As stated in a previous blog, organizations should favor a vendor-agnostic approach for minimizing their integration budget. Lock-in happens when you are using the same vendor for your orchestrator and your inventory. Hence service providers should avoid “vertical integration from vendors” if they want to minimize their automation costs. Otherwise there will be little incentive for your vendors to use a public open API.

The bill should be split

Although integration costs are both significant and “upfront”, it is actually important to properly allocate them. Other automation initiatives will very likely benefit from new APIs. Hence the first couple of automation initiatives should not bear all the costs.

This is why the Program Management Office (PMO) should contribute to the overall TCO calculation. Otherwise the first couple of automation initiatives will be deeply and unfairly penalized from a costing perspective. Thanks to PMO inputs, financial analysts can perform TCO calculation with a global perspective. Making sure integration costs are correctly shared among dozens of automation projects, is probably more important than spending too much effort to figure them out from day one.

Testing is not an option

No need to elaborate about the necessity of using CI/CD for automation projects in this blog. Without a proper automation testing suite, any automation can create more harm than benefits.

DevOps delivery pipeline for telecom
DevOps delivery pipeline

However, is it obvious that a CI/CD roll-out may be costly ? True, virtualization can decrease costs by automating the build of early production labs. However equipment vendors have not virtualized all physical equipment. Not all VNFs perfectly mimic the behavior of their physical counterpart. So one cannot avoid to involve the IT and/or network engineers during lab build out. In addition, DevOps engineers may share early production labs due to the costs of a dedicated lab. In this case some human coordination will ensure minimum project interferences.

Long story short, testing automation activities can lead to significant costs. Yet the finance team should look at them with a holistic approach. As they should do for automation software license fees and integration costs. So eventually these costs may just be a small price to pay to ensure very high quality deliverables.

What are the main takeaways ?

This blog concludes our series about automation TCO based on real-life deployments of our orchestrator solution. Here are our main recommendations:

  • Choose a vendor that minimizes your total automation roadmap costs. Not only for the first couple of initiatives.
  • Run away from vendors offering “vertical integration” and privilege vendor-agnostic solutions if you want to avoid vendor lock-in.
  • Make sure you will benefit from economy of scales for the service modeling. Modeling unit costs should significantly decrease in particular for automation initiatives within the same domain.
  • Develop the right allocation model as most automation costs can be shared.